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SUMMARY 

This report describes a field tmx of a new approach to measuring organi- 
Aiiion^^l productivity and its use in the implementation of feedback, goal setting, 
and incentive systems* 

The productivity measurement system was devaloped for five units in the 
*';aintenMce and supply areas at an operational Air Force base. The productivity 
measures derived from the tystem were used as the basis for formal feedback 
reports given to tiie units monthly for five months. After this, goal setting was 
added to the feedback for five months. Incumbents and supervisors of the five 
units jointly set goals each month* Finally^ incentives were added to the feed- 
back and goal setting. The incentives were time off from work, and units could 
earn either a half or a full day offs depending on their productivity. 

The results showed the approach to be a very effective method of produc- 
tivity measurement and e^ancement. It also has several other advantages such 
as aggregating qualitatively different units into one productivity measure, and al- 
lowing for direct comparison of productivity across different units. Using the 
feedback produced by the system resulted in an average gain in productivity of 
50% over baseline across the five units. When goal setting was added, the mean 
increase was 75% ov^r baseline. When incentives were added to that, the mean 
increase was 76% over baseline. The positive effects lasted over timeg and con- 
tinued after the departure of the research team. The units have continued the 
system on their own, and their managers have requested that it be used in other 
units at Uie base. 

It was concluded that feedback increased prcductivity substantially, goal 
setting increased productivity further, and incentives did not add beyond this. 
The goal setting and incentive conclusions were somewhat tentative due to the 
probable presence of learning and ceiling effects* Different work attitudes un- 
der the program either improved or were unchanged, inclusions about develop- 
ing productivity systems in Air Force environments were discussed. 
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I. INTRODUCTION 



The problem of enhancing productivity has been a universal concern for 
some time. Enhancing productivity has implications for our quality of life, our 
economy, and our competitive position in the world marketplace (Alluisi & Meigs 
1983; American Productivity Center, 1981). In addition, individual organizations 
are continually concerned about increasing their productivity in order to improve 
their operational effectiveness. This concern for increasing productivity is 
shared by the Air Force, and has led the Air Force Human Resources Laboratory 
(AFHRL)^ to explore ways of enhancing productivity thrgugh research and devel- 
opment (R^D) efforts in the productivity domain. 

The present R&D effort was a large-scale field effort aimed at imple- 
menting a new approach to measuring productivity, and using that approach to as- 
sess the impact of feedback, goal setting, and incentives on productivity. This 
technical paper is one of three produced by tlie effort, and describes the entire 
project and its results. A second technical paper from the project, entitled 
"Organizational .Productivity Measurement: The Development and Evaluation of an 
Integrated Approach" (Pritchard, Jones, Roth, Stuebin^, & Ekeberg, 1987a), pre- 
sents a more detailed description of the development and evaluation of the pro- 
ductivity measurement system used in the effort, Altfaough there is some overlap 
between the two papers, this one is designed to present the entire project, 
whereas the second focuses primarily on the productivity measurement system it- 
self. The third paper resulting from the project is quite different in nature. It 
is non-technical and designed to aid operational managers in designing and im- 
plementing feedback, goal setting, and incentive systems. It is entitled 
"Manager's Guide to the Implementation of Feedback, Goal Setting, and Incentive 
Systems" (Pritchard, Jones, Roth, Stuebing, & Ekeberg, 1987b). 

In the present paper, we shall (a) briefly review the literature on feed-- 
back, goal setting, incentives, and productivity meas-urement; (b) describe the 
methods and procedures used in the research; (c) describe the results of the ef- 
fort; (d) discuss and interpret the results; and (e) draw conclusions. 

II. REVIEW OF THE LITERATtJRE 

Feedback 

The positive effect of feedback on performance has become one of the 
most accepted principles in psychology (see reviews by Ammons, 19K; Annett, 
1969; Bilodeau & Bilodeau, 1961; Guzm, Jette, & Katzell, 1985; Ilgen, Fisher, & 
Taylor, 1979; Nadler, 1979; Sassenwrath, 1975; and Taylor, Fisher, & Ilgen. 
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^ : ' i particular interest to organizationftl researchOT is the h«.ost of studies 
that hav« demonstrated the beneficial effects of feedtack on indi«.idual perfor- 
^ organizations (e.g. Dockstader, Nebeker. & Shumate, 1977. Ilgen 
Fisher, ^ Taylor. 1979; Ivancevich & McMahon, 1982.- Prltchard, Sigby. BeiSng' 
Coverdal«. & Morgan. 1981). Although most of the feedback res«^ch has f?-' 
cused oa_ the impact of feedback on individual performance, feedback has also 

Si^ril^; NSlerf^m ^"** "^'^ ^ 

f *u ^^^^^^ technical paper concerns productivity enhancemcait, the review 
of the feedback literature will focus primarily on the type of feedback that has 
been or ^ould be used In organizations. Feedback in organizations consists pri- 
marily oar three t3TPes: (a) feedback to the organization about orfanazational pro- 
cesses ^d attitudes (e.g., survey feedback); (b) performince appraisal feed- 
back; anrf (c) regular feedback about some form of output, such tis units com- 
pleted or number of safety violations per month. This riwew wiHl focus ori- 
marily om this third type of organizational feedback while including relevant con- 
tributions from research on the first two types of feedback where appropriate 



fr°n™ establishing tliat feedback works, feedback reseai— ch has prin- 
cipally fo.llowed three lines of inquiry. The first involves investiga^ting the dif^ 
ferent dimensions of feedback and how a feedback system should be designed 
along tho»e dimensions so that it car: have optimal effict, The se^cond line of 
researchttas been to examine the effects of moderator variables on tthase dimen- 
sions. TOie third area of inquiry has been be to examine the processes or medi- 
ating mec&amsms through wMch feedback affects performance. 

Research Feed back Dimfinsinng 

Ty^es of feedback can be described along a series of dimenslans Differ- 
ent types of feedback may have different effects, and the effects .of feedback 
may be opstimized by understanding these different dimeiisions. Examples of di- 
mensions «f feedback that have been investigated are: lource of tfae feedback 

rt!'L J^"'^oP^ ^^'"'^ * l^^S). Pitive versus negative 

feedback (Fisher, 1979), immediate versus delayed feedback (Be..€son 1973. 
Christian, 1972; Robinson & Kulp, 1970.- Sassenwrath & Younge, 1969)_ knowledge 
of right r^rsus wrong responses (Longstreih. 1970; Merril!, 1970; Wike 1970) 
verbal versus non-verbal feedback (Lair & Smith, 1970; Simpson, 1^^). personal 
versus imp-ersonal feedback (Pritchard, Montagno & Moore, 1978; Wei^ner 1968) 
accurate versus inaccurate feedback (Griswold. 1970), information 'content 
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(Berman, Fraserp & Theious, 1970), and frequency of feedback (Chobbar & 
Wallin, 1984; Ivancevich, Donnelly, & Lyon, 1970), 



Greller (1975) focused on the sources of feedback in organizations and 
proposed a four=dimension feedback source taxonomy. The four dimensions 
were consequences from agents, expressions from agents, task feedback, and 
internal comparisons. Internal comparisons arise from the comparisons one 
makes between one's self and others on the same job. Beyond these four dimen- 
sions, Greller proposed that an evaluative continuum exists along each of these 
four dimensions. When Greller tested this model, he found partial support, but 
found that pKDsitive .md negative feedback were not simple opposites. Building on 
this line of research, Greller and Herold (1977) reported on a factor analysis of 
questionnaire data. They reported the following five factors: negative feedback 
independent of source, positive feedback from superiors, ^sitive feedback from 
non-hierarchical others, internal criteria, and work flow feedback, 

Pritchard and Montagno (15, j) reviewed the literature and listed 14 feed- 
back dimensions, which can be summarized as follows: 

1. Positive versus negative feedback. Feedback can be given for 
correct behavior (or high performance) and/or for incorrect be- 
havior (or low performance). The first is positive feedback; the 
second is negative feedback, 

2* Timing of faadback. Refers to the delay between performance and 
the presentation of feedback, 

3, Specificity, Refers to the molecularity of the behavior for which 
feedback is given, 

4, Evaluative - Non-Evaluative, The feedback may be purely de- 
scriptive or it may contain an evaluative component, 

5, Absolute Comparative, The feedback may be information about 
the recipient's performance (absolute), or may also provide infor- 
mation about performance relative to some set of norms or to the 
performance of other recipients, 

6, Externad - Internid* Refers to the source of the information. In- 
ternal feedback is based on the person's own experience with the 
task. External feedback comes from sources outside the person. 

7, Personal - Impersonal, Concerns the level of personal contact 
between the source and the recipient, 

8, Power of Source. Refers to the degree to which the source con- 
trols the recipient's rewards, 

9, Schedule of feedback. The feedback may be continuous, or it 
may be given at fixed intervals or variable intervals. 
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10. Group = Individu^. The feedback may be provided at the indi- 
vidual or group level, 

11. Comprehensiveness, Refers to the percentage of the recipient's 
role that is covered by the feedback* 

12. Form^ - Iiriformal, Concerns the manner in v^hich the recipient is 
given feedback (e,g,, v^rritten vs. verbal comment). 

13. Public - Private, The feedback may be given to the recipients 
alone or shared with other members of the organization. 

14. Accuracy. Refers to the extent to which the feedback reflects the 
true performance of the recipient. 

Pritchard and Montagno (1978) examined the absolute-comparative and the 
specificity dimensions with simulated inventory control jobs. Non-specific feed- 
back was operationalized as the number of processed orders with one or more 
errors; specific feedback gave information on the type of errors made. Non- 
specific feedback was superior to specific feedback in terms of increasing per- 
formance. Within the non-specific condition, comparative feedback was superior 
to absolute feedback; there was no difference in absolute versus comparative 
feedback within the specific condition. 

Following this same line of research, Pritchard, Montagno and Moore 
(1978) examined the effects of six feedback dimensions in a job simulation ex-- 
periment. Impersonal feedback was found to be superior to personal feedback. 
High specificity was superior to low specificity feedback. Performanca was 
equally improved by group and individual feedback. Public versus private feed- 
back showed no differential effect. The authors concluded that the best combi- 
nation of feedback procedures in this study was impersonal, high specificity, in- 
dividual feedback in either a public or private format. This combination pro- 
duced a 26% increase in quantity and a 27% decrease in errors. 

Continuing this series of feedback studies, Pritchard, Bigby, Beiting, 
Coverdale, and Morgan (1^1) varied the personal-impersonal and absolute-com- 
parative dimensions in a field study. They held constant the remaining dimen- 
sions so that the feedback was specific, individualized, private, comprehensive, 
formal, external, and accurate. The feedback did not contain an evaluative com- 
ponent, but simply reflected the performance levels of the individual incumbents. 
Results indicated that feedback sipuficantly improved performance both in terms 
of quantity and quality. The authors concluded that there were no difference^ 
between impersonal and personal feedback, nor were there any differences be- 
tween absolute and comparative feedback. Thus, they concluded that the choice 
of which type of feedback to use should best be determined by the particular 



14 



circumstances and preferences of the unit to which the feedback would be ap- 
plied. 

Payne and Hauty (1955) reported that specific feedback, which they 
termed "directive feedback,*' increased motor performance over less specific 
feedback* They also fomd that global, comparative feedback, which they called 
"incitive feedback," was superior to global, absolute feedback. 

Dockstader, Nebeker, and Shumate (1977) compared absolute and compara- 
tive feedback. They found that comparative feedback was superior to absolute 
feedback on quantity measures using keypunch operators. They theorized that 
comparative feedback facilitated the setting of implicit goals by the operators. 

In examining the effects of feedback frequency, Chobbar and Wallin 
(1984) varied the frequency of feedback about safety violations in a manufactur- 
ing plant. They found that frequent feedback (once a week) was no more ef- 
fective in increasing safety performance than was less frequent feedback (once 
every two weeks). Examination of the safety performance data indicates there 
may have been a ceiling effect operating so that there was no opportunity for the 
more frequent feedback to show a difference. Both feedback conditions signifi^ 
cantly improved safety performance over baseline levels* 

Greller and Herold (1975) reported that the more proximal the source of 

the feedback, the greater the utilization of the feedback* As one indication of 

this, an individuars own feelings and feedback from the task were reported as 
the most commonly used sources of feedback. 

As a whole, these studies do not identify the characteristics of the optimal 
feedback system* They do show that a variety of feedback approaches work. 
This overall finding is quantified by the meta-analysis of Guzzo, Jette, and 
Katzell (1985). Their results indicate that there is a significant effect on 
performance due to feedback interventions across a variety of situations. The 
authors determined that the mean effect size for feedback interventions 
(including performance appraisal) is .5 standard deviations. 

Moderators of the Feedback-Performance Relationship 

Moderators of the feedback-performance relationship are used to explain 
why a feedback effect may exist, or why it is stronger in one condition than it 
is in another* One of these moderator variables is the initial level of perfor- 
mance or motivation of the recipients* Berkowitz, Le\o^, and Harvey (1957) 
found that initial motivation of the recipients affects the task-oriented behavior 



following feedback. Imtial high motivation individuals exhibited more task ori- 
ented behavior than did low motivation individuals after negative feedback. After 
positive feedback, initial low motivation individuals became more task-oriented. 

Ilgen* Fisher, and Taylor (1?79) reviewed Uie feedback literature and de= 
veloped a m^el of the feedback process. Their model allows for individual 
differences to impact the feedback process at four points: perception of the 
feedbackt acceptance of the feedback, desire to respond to the feedback, and in- 
tended response. They concluded that personality variables create a frame'-of- 
reference that causes the feedback to be interpreted so that it is consistent with 
the recipient's self-image* Thus* persons with high self-esteem rely less on 
feedback from the job environment and more on their own self -perceptions to 
guide their work behavior than do people with low self-esteem (Weiss, 1S77), 
In a related fashion, subjects with an internal locus of control perform better 
when the feedback originates directly from the task itself, whereas those with 
an external locus of control perform better when a person (e,g,, an experimenter 
or supervisor) provides the feedback (Baron, C^wan, & Ganz, 1974; Baron, 
Cowan, Ganz, & McDonald. 1974; Baron & Ganz, 1972)* 

Feedback Mechanisms and Processes 

Research on feedback has also explored how feedback works. In their 
review of the feedback litera^re, Ilgen, Fisher and Taylor (1979) commented on 
the large number of studies relating dimensions of feedback to responses, with 
little attention to the mediating psychological processes. Locke and Bryan (1969) 
noted the same gap ten years earlier: "the question of how KR [knowledge of re- 
sults] facilitates performance has not yet been answered'* (p. 89). 

Several researchers have offered ideas concerning how feedback influ- 
ences performance, Ammons (1956) suggested three mediating effects of feed- 
back which have been elaborated by others (e*g. Becker, 1978; Chapanis, 1964)i 
(a) Feedback informs the recipients about their performance in terms of extent 
and type of performance, (b) feedback rewards or punishes the recipient for ac- 
ceptable or imacceptable performance, and (c) feedback may motivate the recipi- 
ent through intrinsic means« 

Several researchers have examined the motivating effect of feedback. 
Gibbs and Brown (1955) reported that subjects who received absolute feedback 
about the number of units finished increased performance. The explanation was 
that the information provided led the subjects to compete against themselves. 
Chapanis (1964), however, found he could not replicate these results for a simi- 
lar task. 



^cke and his associates have argued that the motivational effect attributed 
to feedback is actually due to goal setting (Locke, Cartledge, & Koeppel, 1968) 
According to this view, feedback will have a facilitating motivational effect only 
when It leads to the setting of a difficult performance goal. This implies that 
feedback is a necessary but not sufficient condition for improved performance 
supporting research for this proposition (Becker. 1978- Erez.' 
& Bob^,'l98f) ' ^ ^"^^ ^^^1' Shaw. Locke! 

It should be noted, however, that Locke views conscious intentions as 
goals. 1 hat IS, If an individual decides that he/she will attempt to produce 100 
umts this is goal setting. Locke's position is that this goal setting must occur 
for feedback to have a positive effect on performance. This is quite different 
than saying there must be a formal goal setting program for feedback to affect 
performance. It is very clear that feedback alone can have powerful effects on 
performance. Thus, the issue that the above literature addresses is whether a 
person must have a conscious intention of performing at a given level in order 
for feedback to work. 

Following the logic that feedback should lead to goal setting, Bigby (1981) 
and Mcrgan (1984) surveyed participants in a field experiment where feedback 
""^ iQQi'? '™P''°''«^ performance (Pritchard, Bigby, Beiting. Coverdale, & Mor- 
gan, 1981). Bigby (1981) reported non-significant decreases in personal goal 
setting from baseline to feedback. Morgan (1984), using a regression analysis, 
found that personal goal setting predicted only one of four performance mea- 
sures. 

Pritchard and Montagno (1978) proposed four mediating mechanisms to ex- 
plain the effects of feedback on performance: (a) improved role clarity (b) ex- 
pectancy-instrumentality-valence effects, (c) implicit goal setting, and (d) direct 
effects on intrinsic motivation. The authors tested all of these mechanisms ex- 
cept implicit goal setting, but found none of the hypothesized changes even though 
performance did increase as a result of feedback. 

The Ilgen. Fisher, and Taylor (1979) model emphasized characteristics of 
the feedback and the recipient which influence several aspects of the feedback 
situation. These intrude the perception and acceptance of the feedback by the 
recipient; the desire of the recipient to respond to the feedback; the intended re- 
sponse of the recipient; and finally, the eventual response of the recipient Re- 
search has supported portions of this model (e.g., Vandaveer, 1981). However 
since the model is intended primarily to explain the performance appraisal 
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feedback process, its contribution is limited in explaining feedback of the type 
with which we are concerned here, 

Nadler (1979) proposed a preliminary model of the effects of feedback on 
group task behavior. Building on propositions by Payne and Hauty (1955) and 
Vroom (1964), and a m^el developed by Hackman and Morris (1975), Nadler 
proposed a model of group performance as it is affected by feedback. Nadler 
reviewed 34 studies, most of which indicated that feedback can improve group 
functioning through a cueing function, a motivational function, or a combination of 
the tv/o. 

Nadler proposed that the variables of aggregation of the feedback (group 
or individual), focus of the feedback (task or process), and the evaluative con-- 
tent of the feedback (positive or negative) exert their effects through the cueing 
or motivating functions but that the effects of these variables are contingent on a 
number of factors. He suggested that the effect of level of feedback aggrega- 
tion on groups is contingent on: (a) the desired impact of the feedback (i.e., af- 
fective, cognitive, or behavioral), (b) the nature of the group task, and (c) the 
characteristics of the group members. He found that group-level feedback en- 
hanced group attitudes and task motivation, but that individual feedback was more 
effective in enhancing individual performance* For the second factor, he found 
that group feedback was more effective when interdependent tasks were per- 
formed by the group and that individual feedback was more effective where 
group performance was simply the sum of individual performances. For the 
third factor, characteristics of group members, he found that affiliation-oriented 
individuals were more responsive to group-level feedback than were non-affilia- 
tion individuals, 

Nadler also examined the effects of the evaluative content of feedback on 
group performance. He concluded that evaluative feedback does lead to "changes 
in member motivation, creation of attributes (including defensive coping) and 
changes in patterns of group interaction or structure," Negative group-level 
feedback is said to have an unclear pattern of effects on group performance, 
since it evokes coping behaviors (Zander & Wulff, 1966) and external attributions 
to the causes of the group's pror performance (Schlenker, Soraci, & McCarthy, 
1976) and may also result in the loweri^ of group aspirations (Dustin, 1966), 

Taylor, Fisher, and Ilgen (1^) reviewed research on individuals' reac- 
tions to feedback taking a control theory approach to explain how feedback influ- 
ences performance. They discussed the issue of discrepancies between employ- 
ees' performance standards and the orgamzation-s performance standards and ar- 
gued that if feedback is to have the impact intended by a source, there must be 



8 18 



high congruence between behavioral standards and performance dinaensions in^ 
eluded in the feedback. 

They suggested four causes of differences between these sets of stan- 
dards. First, feedback recipients may lack standards to guide fteir behavior on 
the job. Second, recipients* standards for behavior may be qualitatively different 
from those of the organization. Third, -nients may have a different priority 
system for standards than the organiEaliOiA uges. Fourth, there may be quantita^ 
tive differences in the level of standards held by the the source and the recipi- 
ent. 

These differences in standards represent a major issue to be addressed in 
developing a feedback system. This is a particularly sensitive issue where the 
feedback system contains standards or an evaluative dimension, A system that 
does not reach some resolution of conflicts between these standards may not be 
very effective* 

Taylor et aL (1984) discussed the effects of feedback on goal levels. 
They cited evidence that goals change in response to feedback in the following 
ways* (a) FeedbacK produces variance in participants' goals* (b) Long-term 
negative feedback may lead to reduced goals; however, the level of sub-goals 
may be increased in response to negative feedback, (c) Positive feedback may 
influence standards by increasing them, decreasing them^ causing them to remain 
constant, or changing their nature. Positive feedback may increase performance 
when ttiere is no obvious payoff associated with surpassing standards as opposed 
to merely meeting them. Standards may also be increased following positive 
faedback (Campion & Lord, 1^2) • Goals may also remain stable after positive 
feedback (a type of **kee^ up the g^^ work" attitude). The nature of standards 
may also change through the alteration of strategies, 

Taylor et aL also looked at the potential behavioral responses to feedback. 
Reminiscent of the cueing function and motivating function of Payne and Hauty 
(1955) and Nadler (1979), they looked at the effects of feedback on the direction 
of behavior* They concluded that negative feedback seems to initiate a search 
for alternative work strategies. In addition, they suggested that negative feed- 
back may decrease effort when the expectancies are low. The authors also ex- 
amined potential responses against the feedback system. Even when feedback is 
perceived as fair, it may provoke a response of "psychological reactance," which 
is a desire to reassert control when one^s freedom of choice is threatened 
(Brehm, 19^; Brahm & Brehm, 1^1), This concept of reactance may help to 
explain why unilateral, critical, or directive performance appraisals and feedback 



tem!T. ^"r^"* "^"^ ""^^ participative apprai.al and feedback svs 

terns (e.g.. Bassett & Meyer. 1968; Kay, Meyer, & French. mS). 

Goal Setting 

m«n.. ^T^'-^- combining goal setting with feedback to e-ihance oerfor- 
manee or productmty have been well established in a wide variety ^ sifuatSns 

moDiey, iv/i), telephone operators (Burke & Wilenx IQAQ'i. i„ • 

Tmi 't'- T'' «"'=-TcrwforwM Tmz:! 

son, 1983) clerical workers and keypunch operators (Dockstader Nebek^^ A 
Shumate 1977; Pritchard, Bigby, Beiting, Coverdale, & Morgt 1^1) wk 
dnvers (LaU>am & Baldes. 1975) and engineers (Ivancevich & McMahon 1982) 
n a n,eta-analy3,s of psychologically based interventions (Guzzo. JetL & Kize, 1 

Goal setting research has primarily followed the same avenues that have 

?ecT r'°hf '"J"' ™ f"*""^ of the goal se^"g e" 

feot, possible moderators, and the process or mechanisms whereby the effe« oc- 

Dimensions of Goal Setting 

t.r-rr.- ^\ "^^^^^Of^d. research has explored the dimensions of goal setting to de- 
termme how goals should be set for maximum effectiveness. These dimensions 
ar^ goa^ diffjculty. goal specificity, and participation in tha settit^ of ^e ^ 
^ !ntt^:r f ---"Sly demonstrate that challenging and spedf^^s'r^: 
suit an bett^er performance than do easy goals, no goals, or vague "do your be^" 
goals (Locke. Shaw. Saari. & Latham. 1981- Tubbs. 1986). Addittona^ 
appears to be a positive, linear relationship between goal difficulty^ perJ^r 
m^ce provided the goal has been accepted (Erez & Zidon. 1^). ' llle ^ al 
19m) concluded that the necessity of participation in the setting of the goa ti 
^ % ^5:,:^^-"^^ a series of studies that demonstrated that wl^ the 
^ ^4^ ifnf^^ " the same for participatively and non=participativeS ^ 
'f^ a V Tubbs. however, in his meta^nal- 

ocke eTal flln \^^'"u Participation. Furthermore, 

-ocke et al. (1981) suggested that participation may have subtle effects on such 
mtcomes as supervisory support, as well as improved understanding of g^s and 
to reach them. In addition, it was found that participativel^^^^^e^ 
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to be set higher thaii non-participatively set goals (Latham, Mitchell, & Dossett, 
1978) and thus t<jnd to lead to higUer performance. 

Moderators of tiie Goal Setting-Performance Relationship 

Several moderator variables have been studied for Aeir effects on the 
goal setting-performance relationship. The follov^ring moderators have received 
empirical support (Tubbs, 1986): 

L Laboratory versus field setting* Goal setting demonstrates a 
stronger effect in the laboratory than in actual organizations* 

2, Presence of and formality of feedback* Specific goals and partici- 
patively set goals are more effective vvrhen feedback is present and 
is formal rather than informal. 

3. Quantity goal versus time goaL Goal setting has a stronger effect 
when the goal is in terms of quantity rather than time. 

Goal Setting Mechanisms and Processes 

According to Locke's (1968) theory, a goal is a mental event that is sim= 
ply what the individual consciously intends to accomplishp The theory assumes 
that conscious intentions regnjlate actionSt and furthermore that goals are the im- 
mediate regulators of action. Because goals are seen as the immediate regula- 
tors of action^ there are no cognitive mediating mechanisms proposed by Locke 
and associates to explain how goal setting works. Instead, the mediating mecha^ 
nisms are seen as actions that occur between the establishment of the goal and 
the final targeted performance level. These mechanisms are indicated below. 

L Direction (of behavior). When goals are established, the goal- rele- 
vant areas of performance are attended to more than are non^goal- 
relevant areas of performance. 

2. Effort. Goals result in higher performance because people work 
harder than if there are no goals, and they work even harder when 
the goals are harder* 

3* Persistence* This mechanism is simply a combination of direction 
and effort over time* 
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Strategy Development. People with goals are likely to develop ac- 
tion plans or to improve strategies for achieving their goals. 



In their review, Locke, et al. (1981) cited evidence supporting these four 
mechanisms. Support appears weakest for the Effort and the Persistence mech- 
anisms, while the Direction and the Straten^ Development mechanisms have been 
more thoroughly researched. 

Naylor and Ilgen (1984) reviewed the mechanisms proposed by Locke et al. 
(1981) and concluded that the mechanisms were quite complex and required 
somewhat ambiguous sub-constructs. Therefore, they proposed mechanisms for 
goal setting that were derived directly from the organizational theory proposed by 
Naylor, Pritchard, and Ilgen (1980), and showed how different types of goal set- 
ting would be explained by the theory. Although testing of some of their propo- 
sitions may be difficult, Naylor and Ilgen make a persuasive argument for em- 
ploying these very precise concepts in explaining how goal setting works. 

Incentives 

The use of incentives to alter motivation and behavior has been researched 
for over half a century (Hull, 1943; Lewin, 1938; Skinner, 1938; Thorndike, 
1911; Tolman, 1932). Incentives have been used in almost every conceivable situ- 
ation where a behavior is to be enhanced. Reviews of incentive motivation theo- 
ries may be found in Campbell and Pritchard (1976); DeLeo (1972); Guzzo 
(1979); and Lawler (1971). Although most formal reward systems in organiza- 
tions employ financial incentives, motivation theories hold that behavior can be 
motivated by a variety of rewards including pay, praise, promotion, recognition, 
time-off, social freedom, self-development, alleviation of boredom, sense of 
accomplishment, etc. 

Although the effectiveness of incentives has been well established in the 
psychological and management literature, only 26% of U.S. workers are covered 
by formal incentive plans aimed at enhancing productivity (Fein, 1976). Incentive 
programs are more prevalent in manufacturing firms (59%) than in non-manu- 
facturing firms (6%) (Rice, 1977). Federal sector organizations, which have 
less freedom in allocating funds for prc^rams such as incentives, reportedly 
spent only .1% of the Federal payroll for cash incentive awards in fiscal year 
1978. Only this small amount was spent, even though the savings to the Govern- 
ment were estimated to average $11 for every $1 spent on incentive awards 
(Spector & Hayes, 1979). 



Taylor (1947) was the first to write about the use of incentives in orga^ 
nizations. He argued that people will work harder when their rewards and pun-^ 
ishments are directly tied to their performance. In addition to the requirement 
of a perceived link between incentives and performance, the rewards must be 
perceived as worthwhile and valuable if they are to be effective incentives 
ti^awler, 1973). Also, the performance goals for achieving the incentives must 
be perceived as attainable if the incentives are to have the desired effect Much 
theoretical and empirical work on expectancy theory has refined these ideas 
(Campbejl & Pntchard, 1976; Lawler, 1973; Mitchell, 1974). 

The better the employee understands the relationship between delivery of 
incentives and performance, the more effective the incentive system will be 
(OpsaM & Dunnette, 1966). A mass of evidence supports this proposition (Arvey 
1972; AtkAnson, 1958; Atkinson & Reitman, 1956; DeLeo, 1972; Evans 1970- Gal- 
braith & Cummings, 1967; Gavin, 1970; Graen. 1969; Hackman & Porter' 1968- 
Jorgenson, Dunnette, & Pritchard, 1973; Lawler & Porter. 1967; Porter & Lawler' 

f ^' * ^"^^^ Georgopoulos, Mahoney. and 

Jones (1957) reported that workers who perceived higher personal productivity as 
a means to increased earnings were more productive than workers who did not 
perceive such a relationship. 

n u^^,-,^^ relationship may be diluted in various situations. 

Campbell (1971) reported a negative relationship between group size and clarity 
of the relationship between pay and productivity. Lawler (1964) reported that on 
the average, managers saw no relationship between their performance and their 
pay, although highly motivated managers did see such a relationship. 

The importance of the perceptions of rewards in determining their effect 
on motivation can be seen in the common example of the supervisor giving praise 
to the subordinate. If the subordinate perceives the praise as condescending, in= 
sincere, or a tactic of ingratiation, the praise may have the opposite of the ef- 
fect intended. 

Incentive Systems in the Military 

Wood, Hakel, DelGaizo and Klimoski (1975) reported on the attractiveness 
and feasibility of using social Incentives in Air Force technical training. Their 
factor analysis revealed that recognition was the most attractive dimension, fol- 
lowed by personal freedom, self -development, social behaviors (such as social 
interaction, being concerned for helping the instructor, etc.), and information 
feedback (such as information on military assipunents, or feedback on strengths 
and weaknesses). The more attractive incentives were those having a direct 
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impact on the trainee; these incentives tended to be more costly and had low ad- 
mimstrative feasibili^. The authors proposed four experimental social incentive 
systems, of which two were individual incentive systems and two were group in- 
centive systems. Within each of these categories, the incentive systems were di- 
vided mto systems administrated by the training instructor or by the instructor 
and the class. The rationale for proposin,^ group incentives was that perfor- 
mance m training was felt to be affected by the development of cohesiveness in 
the class and by class members helping each other reach performance standards. 

Several researchers have reported on the use of incentives to increase 
performance in the military (Cassileth, 1969; Datel, 1970; Datel & Legters 1970- 
1971; Dockstader, Nebeker, & Shumate, ISfiO; Dockstader, Nebeker Nocella & 
Shumate, 1980; Korman, Glickman, & Frey, 1^1; Pritchard, Von Bergen,' & 
DeLeo. 1974). In general, the results demonstrate that the effectiveness of 'in- 
centives IS highly variable. Korman, et al. (1981) found that disposition to enlist 
was not related to the amount of the incentive. Pritchard, et al. (1974) found 
that high-feasibility incentives (letters of commendation, time-off, avoidance of 
work details, choice of uniform, and avoidance of marching in formation) were 
not effective in increasing performance. Pritchard, et al. also found that finan= 
cial incentives were effective only with one of their two experimental groups 
Cassileth (1969) found that incentives of 3-day passes and free movies were ef= 
fective only in increasing performance for high ability trainees; they had no ef- 
fect for low ability trainees. 

These results from the military setting are consistent with those of a 
meta-analysis of intervention programs which found that financial incentives have 
a broad range of effects (Guzzo, Jette, & Katzell, 1^5). These authors re- 
ported an average effect size of .57 standard deviations (d-statistic) with a 
broad^ confidence interval that also included zero (-.10 to 1.24). They concluded 
that "the effects of incentive programs depend heavily on the circumstances and 
methods of applying them and ... on the criterion of productivity" (p. 285). 
When the criteria of productivity were broken down into output (which included 
quantity and quality of production), withdrawal, and work disruption, the effect 
size of financial incentive programs was significantly different from zero for 
the output criteria only. 

Conclusions from the Literature on Feedback. Goal Setting and Incentives 

It is clear from the literature that feedback can have a positive effect on 
mdividual performance. What literature exists also indicates that feedback can 
enhance group performance. However, most of this research has dealt with jobs 
that were quite simple. Usually, everyone did the same work, everyone worked 
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mdependently. there were one or two easily collectible objective measures of 
performance. When we apply these findings to Air Force environments, to^re is 
a very important difference. The vast majority of Air Force jobs ;re much 
more complex than those documented in the literature. This complexity 
because (a) these jobs involve individuals doing many different things (b 
coordmation among individuals is a crucial part of the jobs, and (c) object ve 
measures are not generally available. "ujecuve 

This suggests that to use feedback techniques in Air Force environments 
we mu^ develop ways of providing feedback for these complex jobs. 
means that we must ftrst develop measures of productivity that are applicable to 
complex, interdependent jobs and combine them with techniques of giving feedback 
that are effective for such jobs. " ^ leedback 

Both Jnl^^ general line of reasoning applies to goal setting and incentives. 
Both can be effective. But it is not clear whether the positive effects will also 
occur m a military setting for the type of complex jobs typically found in Air 
Force orgamzations. 

Organizational Productivitv 

Although much has been written on the subject of organizational productiv- 
ity, tnere is little consensus concerning its definition (Tuttle, 1983) This lack 
of consensus is perhaps not surprising, since there are many approaches to and 
perspectives on productivity. However, several major issues are cited tiiroughout 
this literature, and these will be addressed here. 

The first issue to be addressed is whether an efficiency or an ef- 
fectiveness approach should be used in measuring productivity. Both have been 
proposed and used. Efficiency is typically thought of as an output-to=input ratio 
^or example, monthly manufacturing output divided by manpower used to produce 
that output would be an efficiency measure. Effectiveness is usually defined as 
the relationship of outputs to some standard or expectation. For example 
monthly manufacturing output expressed as a percentage of the goal for thai 
month would be an effectiveness measure. In addition, effectiveness usually in- 
cludes quality of the output as well as quantity. 

Efficiency is the more widely used of the two concepts since it is easier 

organizations, industries, and nations (Norman 
& aahiri, 1972). When we hear that productivity growth in the United States has 
declined over the last 20 years (American Productivity Center. 1981). it is an ef- 
ficiency ratio that is being quoted (i.e., price deflated gross nationaf product 
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divided by worker hours). Effectiveness is a much broader concept because it 
includes other factors such as standardst objectives of the organization, expecta- 
tions of interested parties (e,g,, shareholders, regulatory agencies, and cus= 
tomers), and the viability of the organization relative to its competition. Pro- 
ponents of the effectiveness concept ar^e that as the complexity and ambiguity 
of the work increase, effectiveness measures become more important than effi- 
ciency measures (Balk, 1975), Some authors define productivity as a combina- 
tion of efficiency and effectiveness (Balk, 1975; Coulter, 1979; Hanes & Kriebel, 
1971; National Center for Productivity and Quality of Working Life, 1978; Sib- 
son, 1976; Tuttle, 1981), 

A second issue is what perspective should be taken in measuring produc- 
tivity. The literature indicates that the approach to be used in measuring pro- 
ductivity is determined by the perspective of those doing the measuring, Tuttle 
(1983) suggests five perspectives: those of the economist, engineer, accountant, 
manager, and industrial/organizational psychologist. These approaches differ 
primarily in what they are trying to learn from the productivity measurement. 
To the economist, for example, productivity is output divided by associated inputs 
such as labor, capital, intermediate products purchased, and time. This approach 
is typically applied to very macro units such as whole industries or countries to 
measure the economic health of those units* The industrial /organizational psy- 
chologist approach focuses on the the personnel subsystem of the organization, 
and its measures deal with the efficiency or effectiveness of the personneL 
These approaches are quite different. They measure different things, and they 
are used for different purposes. They would also result in very different pro- 
ductivity measurement sy stems. 

The next issue is what measures should be includad in the measure of 
productivity. Clearly, the different perspectives such as the economist's and the 
accountant's have implications for what measures are included. There are, how- 
ever, a variety of other possibilities. Campbell (1977), for example, listed 30 
types of measures that have been used. These included the expected measures 
such as effectiveness, efficiency, profit, and quality, but also included measures 
such as absenteeism, growth, morale, control, internalization of organizational 
goals, evaluations by external entities, and stability. As another example. 
Seashore and Yuchtman (1967) reported a factor amalysis of organizational pro- 
ductivity scores for insurance agencies. They identified ten factors, many of 
which were quite different from those listed by Campbell. They included new 
member productivity* youthfulness of members, business mix, manpower growth, 
and market penetration. 
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The variety of measures that could be included in a productivity measure- 
ment system clearly shows that there is no one set of measures that best defines 
productivity. The diversity of possible measures must be considered in the de- 
sign of a productivity measurement system. 

Apart from the issue of what productivi there is considerable agree- 

ment that a productivity measurement system should include all important aspects 
of the organiMtion's work. If the system is not complete, it could easily en- 
courage neglect by the organization's personnel of those organizational objectives 
which are not part of the measurement system. In such a situation, the actual 
overall effectiveness of the organizat--.n would suffer (Duerr, 1974). 

Another broad issue is whether or not the productivity measurement system 
should provide an overall index of productivity. The use of a single index is be- 
lieved to be very important because of its motivational value. A single index 
provides the members of the unit with a sense of productivity improvement or 
decrement. The single index would also seem beneficial for management 
information purposes. A large nuniber of pieces of information on organizational 
functioning can be very difficult to assimilate and use for making decisions. 
Many approaches to measuring productivity use a single index (e.g. Felix & Riggs, 
1983; Joint Financial Management Improvement Program, 1976; Kim, 1980; 
Peeples, 1978; Tuttle & Weaver, 1986a, 1986b; Tuttle, Wilkinson, & Matthews' 
1985). 

Conclusions from the Productivity Literature 

In summary, there are many approaches, perspectives, and issues relevant 
to productivity measurement. It is tempting to ask: What is the best definition 
and perspective to use in conceptualizing productivity? However, we believe that 
this is the wrong question. Efficiency and effectiveness approaches both have 
their place, as do the different perspectives. How one resolves some of the 
other issues such as whether to use a single index of productivity and what to 
measure depends on the circumstances. The better question is: Under which cir- 
cumstances is which approach most appropriate? 

In order to answer this question, we must determine the purpose of mea- 
suring productivity in the Air Force. The purpose will help determine the cor- 
rect approach. For different purposes, very different approaches would be used. 
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Our ADproa ch To Organizational Productivity 



In the simplest terms, our purpose in measuring productivity is to be able 
to increase it within a given Air Force organization or part of that organization 
It IS our assumpuon that the people in the organization have a great impact on the 
productivity of the organization. Although the technical subsystem is also impor- 
tant, our focus IS not on that part of the system directly but rather on how the 
techmcal subsystem is used by the people. Our position is that to increase pro- 
ductivity we need to increase the productivi^ of the people in the organization 
through increased motivation. With increased motivation, personnel would exert 
more effort and be more persistent in their efforts; they would work more 
efficiently in the sense that their efforts would be more directly related to 
orgamzational objectives; and they would improve their work strategies and would 
use their own and others' time and efforts with less waste. 

Although we believe that both efficiency and effectiveness approaches should 
be included m a productivity measurement system, we believe that the appropriate 
approach for an Air Force productivity measurement system is to first consider 
productivity as effectiveness rather than efficiency. We take this position for 
three reasons. First, effectiveness, with its orientation toward goal attainment 
IS a broader definition of productivity, since it results in a measurement system 
that expresses producUvity in terms of how good that productivity is In 
contrast, an efficiency approach does not carry with it evaluative information on 
what IS a good or bad level of efficiency. The second reason for adopting the 
effectiveness approach is that this approach can more easily generate a mea- 
surement system that combines all aspects of the organization's productivity into 
a single measure. The final reason is that we believe this approach makes it 
possible to get an effectiveness measure and weight it by inputs to get a system 
that combines the best aspects of both the effectiveness and the efficiency ap- 
proaches. 

Given our purpose of increasing productivity in the Air Force, it is criti- 
cal that the measurement system be complete so that increases in measured as- 
pects of the work are not made at the expense of equally important but unmea- 
sured aspects. Finally, the individual measures should be combined into an 
overall measure of productivity for both motivational and informational purposes 
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Research Quastions 



Having reviewed the literature and settled upon an appropriate approach, 
we proceeded toward our specific research objective, which was to provide an- 
swers to the following questions: 

Can the productivity measurement system be effectively developed in 
an Air Force organization? 

Will giving feedback with the productivity measurement system in-- 
crease productivity? 

Will goal setting and incentives increase productivity over feedback? 
Will using the system change work attitudes? 
Will using the system be seen as valuable by the users? 
Will the system operate successfully after the departure of the re- 
search team? 

m, METHOD 



The present effort was conducted at an Air Force base in the southwest^ 
arn United States, and involved sections in Maintenance and in Resource Man- 
agement* The maintenance unit was the ^mmunications and Navigation section 
(Comm/Nav) in the Component Repair Squadron, There were four sections in 
Resource Management, which together comprised the Materiel Storage and Distri- 
bution Branch (MS&D) of tiie Supply Squadron, The four sections were Receiv^^ 
ing, Storage and Issue, Pickup and Delivery, and Inspection, 

The Comm/Nav section repaired a variety of electronie equipment used for 
aircraft communication and navigation. The number of personnel in Comm/Nav 
ranged from 29 to 35 during the course of the project. The MS&D branch was 
essentially the base warehouse. Property was delivered to the warehouse and 
checked in by the Receiving section. Storage and Issue shelved the property and 
retrieved it as it was ordered by units on the base. The Delivery section deliv- 
ered the property to units on base ^at had ordered it. Inspection made sure the 
property was in good condition, and ensured that regulations were being followed 
concerning property packaging, storage, and identification. The number of per- 
sonnel in the MS&D branch ranged from 47 to 54 during the course of the pro- 
ject, with Receiving averaging 15, Storage and Issue 15, Pickup and Delivery 13, 
and Inspection 7, 
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Description of the Productivity Measurement System 



The tiieoretical background for this approach to the measurement of orga- 
nizational productivity stems from the theory of organizational behavior presented 
by Naylor, Pritchard, and Ilgen (1980), In this theory, an individuars role is 
seen as a series of relationships, called contingencies. These contingencies not 
only indicate the important things that the person is expected to do in the Job, but 
also show the relationship between the amount of each of these activities and 
how that amount is evaluated. 

This approach to roles has the advantage of indicating more than the typi- 
cal information present in role specification. The typical information is limited 
to a listing of the important duties to be performed on the job. In the Naylor, 
Pritchard, and Ilgen approach, this information is supplemented by what level of 
performance is expected in each area, and how positively or negatively each level 
of performance is evaluated. 

In essence, we used the Naylor, Pritchard, and Ilgen conceptual approach 
of products and contingencies, and extended its application from individuals to 
organizational units* This application led to the development of a number of 
unique features for a productivity measurement system* We shall discuss these 
later in the report, 

A second source for the development of our approach was the work of 
Tuttle (1981; Tuttle et aL, 1985; Tuttle & Weaver, 1986a, 1986b), In this work, 
also supported by the Air Force Human Resources Latoratory, Tuttle developed 
an approach to measuring productivity which included methods of going from what 
we call products to obtaining objective indicators of how well Uiese products 
were provided. He used a variety of group tectalques to develop products and 
indicators, some of which we also used. 

Steps in the Development of tlie Prcxiuctivity Measurement System, 

The technique used to generate the productivity measurement system is de- 
scribed more fully In Pritchard, et al, (1^7a), but will be summarized here. It 
consists of four distinct steps: (a) identify salient products, (b) develop indica- 
tors of these products, (c) establish contingencies, and (d) ^t the system to- 
gether. 
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Step 1: Identify Pr^ucts. 



Every organization has a set of activities that it is expected to perform. 
These activities result in a set of what Naylor, et al (1980) called products. In 
using the term "product," we mean more than merely a tangible thing that is pro- 
duced. Products can be thought of as the set of objectives that the organization 
is expected to accomplish. The productivity of the organization is a function of 
how effectively the organization generates these products. The first step in de- 
veloping the productivity measurement system is to identify these products. 

To present the steps involved in developing the productivity measurement sys- 
tem, we shall use an extended hypothetical example that will make each step 
more concrete. For this example, we shall use an organization similar to our 
Comm/Nav section. Assume that the unit diagnoses and repairs electronic equip- 
ment. Personnel are responsible for the repair of various items that have mal- 
functioned and have been brought to the shop for repair. Their primary respon- 
sibility is to repair these items as quickly and as accurately as possible. If they 
repair an item and it does not function properly, it is returned for reaccom- 
plishment of the repair. They are periodically inspected by a Quality Control 
function, which determines whether they are accurately following the procedures 
for repair that are detailed in available repair manuals. The maintenance unit 
also has responsibility for conducting on-the-job training, and a person can repair 
a piece of equipment only if he/she has passed the training certification required 
for that piece of equipment. Thus, it is important that a sufficient number of 
people be qualified through training so that all the items can be repaired in a 
timely maimer. 

To develop the system, the first step would be to meet with people from the 
organization to identify the salient products. Let us assume that the following 
products are identified: 

1. Quality of repair. 

2. Ability to meet demand for repairing items (i.e., the organization's 
ability to repair the needed equipment quickly). 

3. Ability to meet training needs (i.e., the degree to which the organi- 
zation meets its on-the-job training needs). 

In actual fact, there might well be more products in such m organization, 
riowever, since our intent here is to explain the logic of the systef", we shall 
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use only these three so that the example remains simple enough for clear pre- 
sentation. 



Step 2: Develop Indicators. 

Once Ihe products are determined, the next step is to develop indicators for 
each of thesm prcrfucts. An indicator is a measure of how well the organization 
is generating tiie product in question. The indicators are determined through in- 
teraction with the people in the organization^ who are asked to think of those 
things which would show how well people in ^e organization are producing their 
products* There may be only one indicator for a given product, or there may be 
more than one. Some indicators will already be available; some will have to be 
newly developed. After the indicators are discussed and refined, the products 
and indicators might lOTk something like this: 

Product 1, Quality of ^ repair. 

Indicator A: Return rate: percentoge of items returned for 

reaccomplishment of repair. 

Indicator B: Percentage of Quality Control inspections passed. 

Product 2. Ability tn meet demand for repairing items. 

Indicator: Number of units repaired divided by total number 

of units brought in for repair. 

Product 3, Ability to meet training needs. 

Indicator: Number of people qualified to work on each type 

of item to be repaired, divided by the number of 
people needed to be qualified. 

As mentioned above, this would not be a complete list of products and in= 
dicators for such an organization, but it does serve to explain the concept used 
for the productivity measurement system. 

Step 3. Establish Contingencies. 

Once the products and indicators have been identified and approved, the next 
step is to establish the contingencies. A contingency is the relationship between 
the amount of the indicator and the effectiveness of that amount of the indicator. 
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Assume that the personnel in the organization said that return rate was an 
important aspect of their work, that to be at the minimum would correspond to 
an effectiveness of -80, and to be at the maximum would be a +70. After the 
other points on the function are identified, this might result in a contingency such 
as that shown in the bottom of Fipire 1* It indicates that going above the neutral 
point of a 10% return rate is positive, but changes are not linear, in that once 
return rate decreases to 6%, further decreases do not represent as great an in- 
crease in effectiveness. Likewise, at the low end, once the return rate gets as 
bad as 14%, they are doing very badly and lower rates are proportionally not as 
bad. 

After this process has been completed for each of the indicators, and once 
all have been scaled and reviewed for accuracy, the contingency set would be 
complete. 

A sample contingency set is presented in Figure 2. For each indicator, there 
is a contingency with its maximum, minimum, and expected level or zero point, 
and a fimction relating it to effectiveness. The first contingency, that for return 
rate, is the same as that shown in the lower part of Figure 1. The second con= 
tingency is for the percent of Quality Control inspections passed. Note that for 
this contingency the expected level is that 100% of these inspections be passed. 
Recall that these inspections are not inspections of the final work but rather, in- 
spections of the process the teclmician goes through in doing the repair. It is an 
index of how well the person is following the manual in doing the repairs. It is 
expected that all repairs will be done in accordance with the manual. Thus, this 
contingency shows that the expected level is doing all repairs (100%) in accor- 
dance with the manual. Since it is not possible to pass more than 100% of in- 
spections, there are no positive values. Thus, anything less than this is below 
expectations, and results in negative effectiveness. Together, these two contin- 
gencies cover Him product of quality of repair. 

The second product has only one indicator^ and hence only one contingency. 
The product is ability to meet repair demand, and the indicator is the number of 
units repaired divided by the number brought in, expressed as a percentage. This 
contingency is roughly linear, except that a value within the 90% to 100% range 
represents a ma^r jump in effectiveness. 

The last pr^uct is meeting training needs, and also has only one indicator - 
number of people q^alifl^ (through training) to repair equipment, divided by the 
number needed, expressed as a percentage. For this indicator it is jwssible to go 
above 100% since, although the org^zation needs only 15 people to be qualified to 
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repair a given piece of equipment, it could actually have more than 15, How-^ 
ever, the contingency becomes flat after 110%, Indicating that having more than 
110% is no more effective than having 110%, Tlie idea is that once there is a 
small excess over the maximum number needed, fcavixig additional trained person- 
nel is not important. 

Two things are particularly noteworthy abo.Jt these contingencies* The first 
is that the overall slope of the function exprasses the relative importance of the 
indicator. For example, the overall slope for th& first indicator (return rate) is 
steeper than for the second indicator (percent of inspections passed)* This re- 
flects the fact that although it is important to pass inspections which show that 
the process of doing the repair was accurata, actually doing the repair so that 
the item functions properly is more important* Second, the contingencies can be 
non-linear. As shall be discussed below, tMs is necessary to accurately reflect 
the realities of an organization's functioning. In many cases, the relationships 
that actually exist are simply not linear* 

It is important to recognize what these proj^arties of the coiitingencies do. 
First, the contingencies reflect differential iin^^rtance* Different things the 
organization does are not equally important* Asp^«cts of the work that are very 
important get steeper contingencies than aspects t^at are less important. For ex- 
ample, in Figure 2, the first indicator for qualify of repair has a range in ef- 
fectiveness from -80 to +70, whereas meeting training needs ranges from -60 to 
+10. This indicates that quality of repair is mor^e important than meeting train- 
ing needs since variations in quality of repair have a greater impact on the ef- 
fectiveness of the organization. Thus, the relati^^e importance of each aspect of 
the work is incorporated into the contingencies. 

Secondly, contingencies allow for the existence of non-linearities between 
amount of the measure and prcrfuctivity. A linear relationship would mean that to 
improve a given amount at the low end of the measure would be as good as im- 
proving that same amount at the high end. However, in the real world it is very 
common for values in the middle range of an indicator to represent large im- 
provements in productivity, and values at the high end to represent a point of di- 
minishing returns. That is, once an organization gets to a fairly high level of 
productivity on one aspect of the work, it is fre^tiently better to try to improve 
something ttat they are not doing as well, rather than continue to improve some- 
thing that is already at a high level. 

For examplep if the repair shop were operating with a very low return 
rate, it might be better to try to improve meeting training needs rather than at- 
tempting to further improve its return rate, T^hus, even though return rate 



27 37 



overall is mora important than training, if raturn rate is good, improving a low 
degree of training readiness can become more important to the overall effective- 
ness of the organization. Another example of this non-linearity would be a situ- 
ation such as that depicted for training needs. Once the organization reaches a 
certain point, further increases are not more effective since all the people that 
are necessary are already trained. The contingencies in our system capture tois 
non-linearity and thus provide a more accurate picture of the organization's func- 
tioning* 

Step 4. Put the System Together, 

Once the contingencies are completed and approved by management, the last 
step is to put the system together. This would be accomplished by first collect- 
ing the indicator data for a given period of time. If the time period selected 
were a month-long period, the data for the four indicators would be collected at 
the end of the month. Then, based on the contingencies, effectiveness scores 
would be determined for each indicator by calculating the effectiveness for that 
level of the indicator. This is illustrated in Figure 3* For example, if the 
maintenance unit had a return rate of 6% in the month of March, examining the 
contingency indicates that such a return rate is associated with an effectiveness 
score of +60 (i*e„ a value of 6% return rate on the horiMntal axis is associated 
with an effectiveness value of 460 on the vertical axis). Continuing this process 
would give an effectiveness value for each indicator, as exemplified in the Fig- 
ure 3, 

Once Ae effectiveness values are determined, they can be summed to obtain 
the overall effectiveness score for products with more than one indicator, as 
seen for the first product. The total effectiveness of the product Quality of Re- 
pair would be the sum of the two indicators comprising that product: +60 for 
return rate and -10 for quality control inspections passed, for a total of +50, 
Next, overall productivity can be calculated by summing all of the effectiveness 
scores. In the example, this Overall Effectiveness score is +^, 

These effectiveness scores have a distinct meaning, in tiiat a score of 
zero means that the organization is meeting expectations; that is, their productiv- 
ity is neither particularly good nor bad. As the score becomes ^sitive, tiiey are 
exceeding expectations. The more positive the score, the more they are exceed- 
ing expectations. As the score becomes negative, ttey are below expectations. 
The closer they are to the maximum possible overall affeetlveness score, the 
closer they are to their best possible productivity. 
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Figure 3. Completed System. 



PRODUCTIVITY: MAINTENANCE UNIT 
DATE: March, 1987 

INDICATOR EFFECTIVENESS 
DATA: MARCH SCORE 

I. Quality of Repair 

A. Return Rate 6% +60 

B. Percent Quality Control 95% -10 

Inspections Passed 

Total Effectiveness: Quality of Repair - +50 

II. Meeting Repair Demand 

A. Percent Demand Met 9056 +10 

III. Meeting Training Needs 

A. Percent Qualified/Need 8096 -40 



OVERALL EFFECTIVENESS = +20 
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This abili^ to simply sum effectiveness scores is one of the major ad- 
vantages of the system. Because the contingencies reflect the relative importance 
and the non-linearity of the indicators, these factors are already incorporated in 
tlie system; thus, a simple summing does indeed reflect tiie overall effectiveness 
of the unit. As will be discussed later, this property also makes it ^ssible for 
the system to be used to aggregate across individual units to determine the pro- 
ductivity of larger and larger units of the orgamMtion. 

Quality of the System 

In order for this approach to be a good measure of organizational produc^ 
tivity» it is obvious that it needs to be accurate. This means several things. It 
means that the listing of products and indicators must be complete* The system 
can easily produce a situation where those toings that are measured are attended 
to, and those that are not measured are somewhat ignored. If there are impor- 
tant functions of the unit that are not included in the pr^ucts, or if important 
indicators are omitted^ tUs uneven attention to important functions can have very 
dysfimctional consequences for the orgamzation. In addition to being complete, 
the system must accurately reflect what the tmit should be doing. This means 
that the products, indicators, and contingencies that are used in the system must 
be the correct objectives, must be accurate measureSt and must be correctly 
scaled on effectiveness. 

Both accuracy and completeness are dealt with in the development of the 
system by having a clear process of approval of tihe system at higher levels of 
the organization. This approval process is made clear from the start. That is* 
at the begiming of the development of the system, all participants are told that 
incumbents and supervisors will develop the products and indicators, which will 
then be presented to higher maMgement for approval. Once higher management 
has approved the products and indicators, the supervisory groups develop the con- 
tingencies, which must also be formally approved by higher management, Al- 
tliough this approval mechanism, and the multiple inputs that it provides, does not 
guarantee completeness and accuracy of the system, it provides a system of 
checks and balances so that the system will be as complete and accurate as pos- 
sible. 

A final point about the quality of the system is that the development of the 
system necessarily introduces subjectivity into the system. Subjectivity is present 
in the listing of the prwiucts and indicators, and especially in the ratings that are 
used in the contingencies. Subjectivity is present, but this is not necessarily a 
problem. The elements of the system— products, indicators, and contingencies- 
are actually statements of policy* As a whole, they say (a) what is important to 
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the functiomng of the unit, (b) the level of output that is expected (the zero 
pomt . (c) how good other levels of output are. and (d) the relative importance 
of different ypes of functions for the unit. These determinations represent 
po icy. and pohcy is a subjective thing. A manager's primary responsibility is to 
set policy, m the sense that he/she must determine the priorities for resource 
allocation withm his/her unit. What this approach does is to reduce ambiguity in 
^hcy and priorities by formally discussing them, quantifying them, and subjecting 
them to formal review and approval by the management of the organization. 

Priorities 

* .J^T T "^^^ °f productivity measurement sys= 

tern that should be described; the first is the system's capability to generate unit 
priorities. 

The system offers a way to develop a clear set of priorities for improving 
productivity. Recall from Figure 3 that for a given time period (e.g.. a month) 
the system presents the actual amount of each indicator achieved for that period' 
and the effectiveness levels of those amounts of the indicators. It would be a 
simple matter to look at the contingency for each indicator and calculate the 
effectiveness gain that would occur if the unit went up one increment on each of 

/ ^^^^ ^"""^"^ "-''^ example, if the unit had a return rat. 

of 6% in March, as is Indicated in Figure 3, for them to go to the next level up 
U 4% return rate) in April would mean an increase in effectiveness from +60 to 
+^5, for a gam m effectiveness of +5 units. This could be calculated for each 
indicator. Once it was calculated, one could rank order the changes from high= 
est to lowest gam. Such a listing for our example is shown in Figure 4. 

This information communicates exactly what should be changed to maxi- 
mize productivity. The example shows that the best thing the unit can do is to 
focus their efforts on meeting repair demand if they want the highest increase in 
productivity. Once tJus is done, or if it is not possible at the present time th- 
next best thing they could do is to improve training so that more people' are 
qualified. Improvir* quality control inspections and Improving return rate are the 
least important in terms of increasing productivity, with quality control inspec- 
tions slightly more important. 

Thus, the system can generate a set of priorities that unit personnel can use 
to guide their efforts to increase productivity. This would aid in decisions about 
resource allocation, and where to focus to identify barriers to productivity 
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Figure 4. Priorities for Increasing Productivity 



PRIORITIES FOR; APRIL. 1987 

GAIN m 

CHANGE EFFECTIVENESS 



Percent Demand Met from 90% to 1CMD% +90 

Percent Qualified (Training) from 80% to W% +20 

Percent Quality Control Inspections Passed 

from 95% to 100% +10 

Return Rate from 6% to 4% +5 



Aggregation Across Units 



A second unique feature of the productivity measurement system is the 
ability to aggregate across organiEational units. It is quite valuable to have a 
productivity measurement system for a given unit, or several units. It would be 
even more valuable if one could aggregate the measurement system from the sev- 
eral different units into one measure applicable across all the units. For exam- 
ple, if a branch were composed of four separate units, it would be valuable to 
have a measure for each unit, and to be able to combine those four measures 
into a single measure for the entire branch. In most productivity measurement 
systems this is not possible, since the measurement system varies from unit to 
unit. An advantage of our approach is that it is possible to do such across-unit 
aggregation. Each unit is measured on a common metric: overall effectiveness. 
The method for combining their individual effectiveness values to determine the 
overall effectiveness across units merely involves a simple additional scaling step 
in the development of the contingencies; this procedure is explained in detail in 
the report on the productivity measurement system (Pritchard, et al., 1987a), 

This rescaling process has the effect of adjusting the effectiveness scores 
of the different units in the branch for any differences in im^rtance of the dif- 
ferent units. Once it is finished, the overall effectiveness values from the dif- 
ferent units can simply be summed to determine the overall effectiveness of the 
entire branch. This approach to aggregation can be extended to larger and larger 
units, so that, if desired, a single index of the productivity of the entire organi- 
zation can be developed* 

System Development 

To actually develop the praJuetlvity measurement system at the base, 
meetings were held jointly with incumbents and supervisors. First products and 
then indicators were developed. Once the products and indicators had been fi- 
nalized by group consensus, the next step was to obtain approval of the products 
and indicators from higher management, Meetii^s were held with higher man- 
agement where tiie products md indicators were presented, discussed in detail, 
and, with some revisions, approved. As an example, the final set of products 
and indicators for ^mm/Nav are presented in Figure 5* 

The next step was the development of the contingencies. Meetings were 
again held with incumbents and supervisors of each unit. First, the maximum 
and minimum indicator levels were established^ then the zero points were gener- 
ated. These decisions also took considerable time, resulted in considerable ini- 
tial disagreement, and were done over multiple meetings* These meetings were 
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Figure 5. Comm/Nav Products and Indicators, 



Product 1, Equipment Repair 

Bounces: Percantaga of repaired equipment that did not function 
immediately after installation. 

Percent QA (Quality Assurance) inspections passed. 

AWM: Number of units awaiting maintenance, 

AWP: Number of units awaiting parts. 

Demand Met: Percentage of equipment brought in for repairs that 
was actually repaired. 



Product 2, Training 

STS Tasks C^'impleted: Mean number of standard (more basic) 
training tasks completed for persomel in training. 

Percent Qual Tasks Completed, Comm: Mean percent of advanced 
training tasks completed for personnel repairing 
communications equipment. 

Percent ^al Tasks ^mpleted^ Nav: Mean percent of advanced 
training tasks completed for personnel repairing navigation 
equipment. 

Scheduled Training Tasks Overdue: Total number non-technical 

(e.g. military) training requirements not met on time for all 
shop persoimel. 



Product 3. Other Duties 

Mobility Equipment* Number of pieces of equiDment used for 

mobility exercises that were not calibrated by the shop on 
schedule. 

PRffiL Overdue: Number of pieces of shop calibration and test 

equipment that were not calibrated by the shop on schedule. 

Percent 349 Errors: Percent of errors on a major manpower 
documentation fofm. 

Missed Appointments: Number of formal on-base appointments 
missed « 
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held until consensus was established. The contingencies were then presented to 
management for discussion and approval. 

Treatments 

Once the productivity measurement system was- completed, the next phases 
of the research involved instituting the interventions of feedback, goal setting 
and incentives. To do this, a baseline was first established. This amounted to 
a period of eight months for MS&D and nine months for Comm/Nav where indi- 
cator data were collected, but no feedback was given to the units. These data 
provided a baseline against which the effects of the interventions could be evalu- 
ated. 

The schedule was as follows: First, each of the five units developed its 
productivity measurement system. This was followed by the baseline period 
Next, feedback was given to each unit for five months. Goal setting was then 
added to feedback for each unit, and continued for another five months Finally 
incentives were added to feedback and goal setting in each unit for five months. ' 

Feedback 

Productivity feedback consisted of formal computer-generated reports that 
were given to the personnel in each section monthly. To develop these reports, 
we first produced examples of what the basic report would look like and asked 
for supervisors' thoughts on how best to present the material for maximum clar- 
ity. We also proposed some other information that they might find useful, and 
asked for their suggestions on items to be included. After several revisions, a 
final version of the productivity feedback report was developed. An example 'of 
the monthly report for the Comm/Nav shop is presented in Figure 6. 

The first page of the report provides the basic productivity data. It 
shows the products and indicators, the indicator data for that month, and the ef- 
fectiveness score associated with that level of each indicator. The lower portion 
of the page shows the total effectiveness for each of the products and finally, 
the overall effectiveness for the shop. The second page of the report adds in- 
formation to the basic data. The top half of the report shows the change in 
productivity from the previous month to the current month. The indicator data 
and effectiveness scores for both the previous month and the current month are 
shown, as are the changes in effectiveness from last month to the current month. 
This part of the report was requested by shop personnel to aid them in di- 
agnosing areas where they were increasing or decreasing in productivity. 
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Figure 6, Sample Feedback Report. 



PRODUCTIVITY REPORT 
COMM/NAV SHOP 
CRS MAINTEN^CE 



INDICATOR AND EFFECTIVENESS DATA FOR MARU-^ 



OTJICATOR EFFSCIT^N^S 

mPICATOR DATA SCORE 

EQUIPMENT REPAIR 

BOUNCES 3.1 76 

% QA INSPECTIONS PASSID 90.9 30 

AWM 13.5 80 

AWP 39.6 29 

DEMAND MET 91,7 63 

TRAINING 

STS TASKS COMPLETED 8 35 

% QUAL TASKS COMPLETEDi COM 69.5 72 

% QUAL TASKS COMPLETEDi NAV 56.8 68 

SCHED TRAINING TASKS OVERDUE 0 10 

OTHER DUTIES 

MOBILITY EQUIPMENT OVERDUE 0 25 

PMEL OVERDUE 0 25 

% 349 EraORS 1 40 

MISSED APPOINTMENTS 0 10 



BFFmyTrTENmB 

TOTAI^ SCO^ 

EQUIPMENT REPAIR 278 
TRAINING 185 
OTHER DUTIES 100 

OVERALL EFFECTIVENESS 563 
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Figure 6. (Concluded) 



EFFECTIVENESS CHANGE FROM FEBRUARY TO MARCH 





INDICATOR 




INDICATOR 








DATAi 


EFF. 


DATA- 


EFF. 


CHANGE 




FEBRUARY 


SCORE 


MARCH 


SCORE 


IN EFF, 


BOUNCES 


2.8 


81 


3.1 


76 


-5 


% QA INSPECTIONS 


91.7 


34 


90.9 


30 


=4 


AWM 


15.58 


72 


13.5 


80 


8 


AWP 


40.6 


27 


39.6 


29 


2 


DEMAND MET 


91.5 


59 


91.7 


63 


4 


STS TASKS COMPLETED 


9 


52 


8 


35 


= 17 


%QUAL TASKS^COMM 


68.6 


72 


69.5 


72 


0 


%QUAL TASKS^NAV 


59.5 


71 


56.8 


68 


=3 


SCHEDULED TRAINING 












TASKS OVERDUE 


0 


10 


0 


10 


0 


MOBILITY EQUIPMENT 












OVERDUE 


0 


25 


0 


25 


0 


PMEL OVERDIJE 


0 


25 


0 


25 


0 


% 349 ERRORS 


2 


27 


1 


40 


13 


MISSED APPOINTMENTS 


0 


10 


0 


10 


0 



CHANGE TOTALS EQUIPMENT REPAIR 5 

TRAINING ^20 

OTHER DUTIES 13 

OVERALL IFFECTIVENESS -2 



POTENTIAL EFFECTIVEI^SS GAWS FOR NEXT MONTH 





FROM 


TO 


GAIN 


BOUNCES 


3,1 


0.4 


17 


QA INSPECTIONS 


90.9 


100 


45 


AWM 


13.5 


0 


15 


AWP 


39.6 


22.6 


48 


DEMAND MET 


91.7 


95.2 


37 


STS TASKS COMPLETED 


8 


9 


17 


%QUAL TASKS COMPi COMM 


69.5 


76 


0 


5IQUAL TASKS COMP: NAV 


56.8 


62.8 


4 


SCHED TRNING TCKS OVERDUE 


0 


0 


0 


MOBILITY EQ dVERDUE 


0 


0 


0 


PMEL OVERDUE 


0 


0 


0 


349 ERRORS 


1 


0 


15 


MISSED APTOINTMENTS 


0 


0 


0 
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The bottom half of the page is tiie information on priorities for incraasing 
productivity. For each indicator Uiere is a column labeled FROM, TO, and 
GAIN* The FROM column is the amount of the indicator done for the current 
month. The TO column is the amount of the indicator that represents an in- 
crease of one unit on the contingency, and the GAD^ column indicates the gain in 
effectiveness that would be achieved by such an increase. For example, for re- 
pair demand met, if the shop vvent from their March level of 91,7% to 95,2% in 
April, effectiveness would increase by 37 points. Examination of the GAIN col- 
umn indicates that for the next month, the shop would increase their productivity 
the most by focusing on Quality Assurance Inspections, and the number of imits 
Awaiting Parts (AWF), These show potential gains in effectiveness of 45 and 
48 respectively. It would not be useful to devote attention to training in Comm 
Qualification Tasks* trying to further decrease Overdue Scheduled Training 
Tasks, or any of toe other indicators that have a gain value of zero or near 
zero. This information can therefore serve as a basis for determining priorities 
for the next month. It suggests those areas where the unit should focus to pro- 
duce maximum gains in effectiveness. The feedback rejpwrt for MS&D was 
similar to this Comm/Nav report. A copy of this report is included in Pritchard, 
et aL (1987a). 

The calculation of the GAIN amount is based on the amount of increase in 
effectiveness that would occur with an increase of "one unit" of the indicator. 
The size of a one tmit increase was determined from the indicator values used 
in the contingencies. If the indicator values in a contingency were 2%, 4%, 6%, 
8%, etc, the size of e one unit increase for that indicator was 2%. If the indi- 
cator values were 50, 60, 70, 80, etc., tiie size of a one unit increase was 10, 
The contingencies were originally developed so tiiat the number of increments for 
ttie different contingencies was as equal as possible so that a "one unit" incre- 
ment was rougUy comparable across the different contingencies. Once the size 
of the "one unit" increase was determined for each contingency, the TO figure 
was calculated by adding the one unit increase to the actual value of the indicator 
for the preceding month. If tiie last month's indicator level was 83.6 and the 
size of one unit was 10, the FROM value would be 83,6, the TO value would be 
93.6, and the GAIN value would be determined by what the contingency indicated 
as the gain in effectiveness if tiie unit went from 83.6 to 93.6 on that indicator. 

There was one special circumstance that had to be dealt with using this 
approach. It was possible for the TO value to be higher than the maximum 
value of the indicator* This occurred when the imit was already high on that in- 
dicator and increasing "one unit" would put them over the maximum. It also oc- 
curred occasionally if the unit was already over the maximum on that indicator. 
This was dealt with by using the maximum possible effectiveness value for the 




indicator as the upper limit in effectiveness. In other words, if the effective- 
ness value for being at the maximum of the indicator was +75, this was the 
maximum effectiveness score that could be gained from that indicator If the 
umt was near tiie maximum already with, for example, a past month's indicator 
w'^Ih K f i i ^ effectiveness score of +73. the most they could improve 
would be to the value of the ceiling. +75. for a maximum gain of only +2. 

One fe^ature of the system is that it allows one to directly compare the 
productivity of very different units. This feature was very Important to the su- 
pervisors and managers of the MS&D branch since it allowed them to compare 
fte four sections of the branch. In order to make this comparison, we first 
determined the maximum possible overall effectiveness for each section This 
was done by determining the effectiveness value for the maximum possible value 
of each indicator, and summing these effectiveness values. The resultaht score 
represented the effectiveness value that would occur if the unit was doing as 
well as It was possible to do on every aspect of their work; in other words 
ter maximum possible overall effectiveness. Recall that these maximums were 
developed by consensus among the supervisors of the units, and were discussed 
and approved by management. Thus, they should represent realistic maximums 
and the effectiveness scores represent the value of the maximum contribution 
each of the umts could make to the organization. 

Once the maximum possible effectiveness was calculated, the actual 
monthly overall effectiveness score for each section was expressed as a percent- 
age of maximum possible effectiveness. This percentage of maximum effective- 
ness was the measure by which each unit was compared to the other These 
data were included in the monthly feedback report for each section of MS&D. 

The feedback report was generated each month for 15 months for each of 
the five umts. It was presented within three workdays after the end of the 
month, and a copy was given to each individual in the chain of command, from 
the section supervisors to the Deputy Commander. A copy was also posted in the 
working area of each section so incumbents could review it. In addition graphs 
were posted in the work area and updated each month: one for overall effec- 
tiveness, and one for each indicator. These graphs allowed unit personnel to see 
changes m effectiveness over time. As one might imagine, both the feedback re- 
port and the graphs generated considerable interest when they were posted each 
month. 



Once the feedback reports were circulated, a meeting was held with 

cumbents, supervisors and, for MS&D. a management representative to review tlie 
feedback report for the month. Areas of improvement were noted and areas of 
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decrease discussed. Reasons for the improvements or decreases were consid- 
ered, and any longer-range trends were noted. This meeting also served as a 
basis for planning priorities for the next month, and for altering work strategies 
to improve productivity. 

Goal Setting 

After five months of feedback, goal setting was added to the receiving of 
feedback reports. The first step in implementing goal setting was to train the 
personnel in setting productivity goals. The nature and process of goal setting 
was explained to them, and the im^rtance of their participation was stressed. 
They were told the importance of setting a difficult but attainable goal. Prior to 
the start of the goal setting program, a meeting was held with supervisors and 
incumbents to explain how the program would work. They were told to discuss 
goals among themselves for a few days. After this time, the first goal setting 
sessions were held. These were attended by members of the research team to 
facilitate the process, and especially to ensure participation and to encourage 
them to set reasonable goals. 

One issue that became quite significant was whether the goals would be 
"reportable" or "non- reportable." Unit personnel were reluctant to set goals that 
represented difficult but attainable goals, if the goals were to be reported to 
higher management. They feared that they might be held to whatever goals they 
set and would thus receive a negative evaluation if these goals were not met, 
even if productivity had actually increased. It was decided that public or re- 
portable goals would be set at a level that would be low enough to ensure that 
the unit would exceed them. Such goals would have little motivational force, and 
fail to have a positive effect on productivity. Thus, the goal setting system was 
designed to be non-reportablei that is, the goals would be for section use only, 
and would not be communicated to higher management. Section incumbents and 
supervisors set the goals, members of the unit knew what the goals were, but 
the goals were not communicated beyond branch supervisors. Upper management 
agreed to the use of non- reportable goals. They felt that if productivity contin- 
ued to be as high as it had been during feedback, or if it improved, they did not 
need to know the unit goals that helped produce such results. 

Comm/Nav and each of the four units of MS&D set goals for themselves 
in terms of their overall effectiveness for the following month. Goals were set 
jointly by incumbents and supervisors. When the month-end feedback report was 
produced, the unit personnel noted whether or not they had attaint the goal, and 
discussed reasons why the goal was or was not met. The goal was then re- 
viewed and reset for the following month. 



40 50 



Incentives 



After five months of feedback plus goal setting, incentives "were added as 
the final treatment. Following considerable discussion and input from samples of 
incumbents up to senior management, the incentive chosen was time-off from 
work. Specifically, unit personnel were given a half day or full day off if the 
unit achieved the level of productivity needed to qualify for the incentive* This 
incentive seemed powerful for the personnel involved and was feasible to imple- 
ment. 

Several other issues had to be dealt with in designing the incentive system. 
One was the number of levels of pr^uctivity that would be defined as resulting 
in different incentive levels. At one extreme is a system where there is a sin- 
gle level of productivity ttat, if reachedi leads to an incentive* At the other ex=- 
treme is a piece- rate- type system where each increase in productivity leads to an 
increase in ttie amount of incentive. The advantages of the single-level approach 
are that it is simple to administer and can be used with any ^e of incentive. 
The advantage of the multiple-level approach is ftat no matter where the unit's 
productivity is, there is still an incentive to increase productivity further. With 
the single-level approach, once the level of productivity that produces the incen-- 
tive is reached, there is no incentive to increase pr^uctivity further. 

Another design issue concerned the effects of asking the units for higher 
and higher levels of productivity. At the time tiie incentive system was being de- 
signed, all the imits had greatly increased their productivity. In one sense, the 
system had asked them to increase their productivity during feedback and again 
during goal setting* If the incentive system asked for even greater improvement 
before any incentives were awarded, we felt there was a possibility of a nega^ 
tive reaction to the program. 

There was also an issue of setting the incentive levels so that they were 
equitable across units. For Comm/Nav this was not a problem since they had 
little contact with MS&D. However, the four sections of MS&D were in constant 
contact with one another, and the system had to be designed such that no one 
section had an easier time obtaining time-off tiian mother. If the participants 
judged the system to be inec^itable, it could not be effective. 

Finally, there was the issue of whether to use branch-level incentives in 
addition to the section-level incentives in MS&D, The four units of MS&D had to 
cooperate extensively to get the work done. It would have been ^ssible for a 
given section to do tiungs that maximized their overall effectiveness but which 
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caused problems for the other sections. Therefore, the incentive system had to 

have a mechamsm for fostering cooperation among the units. 

The incentive system was designed with these issues in mind. The sys- 
tem used two levels of the incentive, a half day off and i full day off. 

the n..^'""^^^'i '^^f"^ ^ °" ^^'^ effectiveness exceeded 

the mean overall effectiveness for the five immediately preceding months If 
ftey exceeded tins level by a meaningful amount (as dete^ned jointly by^Lm- 
bents. supervision, and management), they received a full day of f It was de- 
cided that ft,r the full day off to be awarded, overall effectiveness must ^ceed 
the mean of the last five months by 5%. Since Comm/Nav was very close to 
fteir m^imum possible effectiveness by this time, a 5% increase was judged to 
be a substantial improvement. One problem with this system was that if pro- 
ductivity continued to increase, the mean of *e last five months would contLe 
o rise. Tins would mean that it would be more and more difficult for the unit 
to achieve the productivity level necessary for incentives. Eventually, continued 
mcreases m productivity would make it impossible to get any incentives. To 
deal with this problem, a maximum was set on the overall effectiveness level 
necessary to get tiie incentives. For the half day oit, the maximum level was 
set at m% of their highest possible overall effectiveness; for the full day off 
the maximum was set at 94%. * 

MS&D also had two levels of the incentive, but with a different structure 
A given section received a half day off if its overall effectiveness exceeded the 
mean overall effectiveness of the five highest productivity months prior to the 
start of incentives. In practice, this meant that they would receive the incentive 
If their productivity exceeded the mean productivity under goal setting There 
was also a branch-level incentive of an additional half day off to be given to 
ea^h section if the branch reached its incentive level on overall effectiveness 
This productivity level for the branch to get the additional half day off was es- 
semially the sum of the section-level goals. That is. the individual section in- 
centives were given when the section's productivity continued at a very high lev^l 
lo get the branch incentive, productivity had to be maintained at this high level 
for each section. This was done since it was felt that several of the sections 
were already performing very near the maximum possible level, and asking them 
to go above this to get the full day off as in Ojmm/Nav was unreasonable If a 
section reached its incentive level and the branch reached its level, that section 
received a full day off. If the section reached its level but the branch did not 
tte section received a half day off. If the section did not meet its incentive 
level but the branch did. the section received a half day off. 
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Thus, the system dealt with the issue of asking for higher and higher 
productivity in Comm/Nav by having one level of the incentive for continuing to 
maintain previous high levels of productivity, and a second level of incentive for 
exceeding that pr^uctivity. For MS&D, tius issue was dealt with by requiring 
that a section maintain its high level of productivity for the half day off, and 
that all sections maintain high productivity for tiie full day off, A branch-level 
incentive was used in MS&D in addition to the section-level incentive. This in- 
centive was structured in such a way that sections were encouraged to cooperate. 
To get the greater incentive, the entire branch had to do welL A section could 
have a bad month and still get a half day off if they could help the rest of the 
sections pull up the branch total. Another possibility was that a section that was 
doing extremely well would help another section so that both would get time off. 

The issue of equity across sections in MS&D was dealt with by first ex- 
plaining the issue of equity to the personnel, and then having Incumbents, supervi- 
sors, and management of the different sections jointly determine what levels of 
productivity should be reached to obtain incentives in each of the sections* 

During the incentive treatment, the units continued to set tiieir own per- 
formance goals and review them each month. We felt that perhaps after the in- 
centive treatment started liiey might simply set their goals equal to the level 
needed to receive time-off, and thus, continuing goal setting might be unneces- 
sary. However, the goal setting was continued so that the individual units could 
set different goals if they so desired, and so that the integrity of the experi- 
mental design was maintained. Specifically, by allowing units to continue their 
own goal setting, we could assess the incremental effects of incentives when 
added to goal settii^* If goal setting had been discontinued, assessment of this 
incremental effect would not have been ^ssible. 

IV, RESULTS 

The results of the effort will be presented in five sections: (a) de- 
velopment of the productivity measurement system; (b) effects on productivity; 
(c) results for Ae feedback, goal setting, and incentive interventions; (d) attitude 
data; and (e) effects after tiie departure of the research team. 

Development of the Productivity Measurement System 

There were a number of results— both qualitative and quantitative^ — perti- 
nent to the development of the productivity measurement system. 
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One qualitative result was the change in the attitudes of the unit personnel 

during the development of the system. When we first started working with 
them, their attitudes toward the project were mixed. Although some unit person- 
nel were positive, others were more skeptical. Those with the more negative 
attitudes felt that the researchers were non-military outsiders who did not know 
the details of their work situation but were going to impose a management sys- 
tem about which these personnel had reservations. Though these people were 
willing to cooperate, they definitely had their doubts. Through the course of the 
system development, their attitudes changed dramatically. By the time system 
development was completed, most unit personnel had positive attitudes toward the 
effort. They were solidly behind the system, felt positive toward the re- 
searchers, and were quite disappointed that they had to wait through the neces- 
sary months of baseline before feedback could begin. 

A second qualitative finding deals with interpreting the strength of the pro- 
ductivity results. Development of the system, by its very nature, led the partici- 
pants to examine unit objectives, suggest possible measures of these objectives, 
and evaluate their productivity expectations and limits. This process led unit su- 
pervisors to see numerous places where improvements could be made in the op- 
eration of the units. They began to implement these changes. This created a 
real dilemma for the researchers. Although it was certainly worthwhile for the 
units to improve their effectiveness because of the development of the system, 
this improvement was occurring prior to the start of our baseline. If. because 
of this, the baseline period showed higher effectiveness than it otherwise would 
have, there would be a decrease in the size of any feedback effect. There was 
little the researchers could do about this dilemma. The units felt strongly that 
such changes should be made, and made them. They felt these changes were 
increasing their effectiveness, and this indeed seemed to be the case, based on 
what little data were available at the Ume. Interviews with supervisors indicated 
that they believed a substantial portion of this improvement was due to the pro- 
cess of developing the productivity measurement system. This suggests that the 
improvements in productivity that were evidenced in the interventions were, in 
fact, underestimates of the overall impact of the development and introduction of 
feedback from the productivity measurement system. 

The reliability and validity of the system were also assessed during devel- 
opment of the system. Reliability was assessed by examining interjudge 
agreement on the contingencies. The ^mm/Nav shop had two shifts in 
operation. Personnel from both shifts were involved with the development of 
products and indicators. To assess reliability, we developed two independent sets 
of contingencies; this produced two effectiveness scores for each value of each 
indicator: one set from the day shift and one set from the night shift. 
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Qjrrelatioii3 calculated between the two sets of values for each contingency 
ranged from a low of M to a high of ,99, with an average of ,95. Thus, the 
reliability of the contingencies as measured by intarjud^Ige agreement was quite 
high. 

The validity of the system was evaluated using fSve different productivity 
scenarios of hypothetical indicator data developed for C^Mim/Nav. This was done 
by selecting a reasonable value for each indicator in sucfa a way that the differ- 
ent scenarios varied as to their overall effectiveness. Although the overaU 
fec^iveness of each of the five scenarios varied, the differences were not so 
larite as to be completely obvious. Also, the changes In indicator values varied, 
but not always in the same direction. That is, although mhe overall effectiveaess 
went up in a given scenario, some indicator values went rfown, while others went 
up. Six (^mm/Nav supervisors were then given the iiii^icator data on the five 
scenarios a»d asked to rank the scenarios as to their o^i^erall effectiveness. If 
the system accurately reflects relative importance, having supervisors rata tlie 
scenarios without knowing the scenario overall effectivei^ess scores should pro^ 
duce ratings which are highly correlated with overall effectiveness as calculated 
by the system* 

Thes^ ratings were done approximately two month_s after the developnient 
of the system had been complated, but before any pr^iductivity feedbacK lad 
started. Results showed a correlation of 1.0 between ea^h supervisor's rankijigs 
and the ovsrall effectiveness score calculated by the sy^^tem. This constitutes 
additional evidence for the validity of Hie system. 

Finally, data were also collected on subjective reactions to the productivity 
measurement and feedback system. All incumbents anS supervisors (N ^ % 
which includes some supervisors not technically part of the units, but in their 
chain of coammand) were surveyed after several months of experience with ihe 
feedback system. The survey asked for their reactions to different aspects of 
the feedback system. The survey items consisted of 5-i^oint Likert scales with 
response formats ranging from Strongly Agree to Strongl— y Disagree. Responses 
to these items were imiformly positive. Table 1 presents the items and the per^ 
centage of respondents who agreed or strongly agreed a^^d the percentage wlio 
disagreed or strongly disagreed* After reverse-scoriitf the ne^^tively phrased 
item (#12), the mean percentage of res^ndents across ^mll items who Strongly 
Agreed or Agreed was 64.7%, while the corresi»nding pes^centage who Disagreed 
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Table 1. Subj^tive Evaluatiorai of the System 



PER^CENT PERCENT 

AGR^E OR DISAGREE OR 

STRONGLY STRONGLY 
ITEM A^REE DISAGREE 

1. The faedback systam tells me how 

good a job I am doing* ^54 4 

2. The faedback system tells me how 

good a job tiie section is doing. ^37 1 

3. The feedback system helps me see 

the section-s priorities* K77 6 

4. The feedback system 'helps the 

section be more pr^uctiva, ^51 4 

5* A system like this would help other 

Air Force bases be more productive, ^52 4 

6, The feedback systam is clear and 

understandable* ^S8 7 

7* It was worth the effort to devalop 

the feedback system, 10 

8. It was worth the effort to keep the 

faedback system in operation, ^52 10 

9. The information about section 
performance that goes into the feedback 

system is accurate* ^32 13 

10. The feedback system gives a good 

measure of productivity, 13 

11. Overall, I like the feedback system. ^S2 9 

12. I would prefer not to have this 
faedback system at the next organization 

I work in, n3 54 

13. The feedback system is a better 

way of measuring productivity than what . 

the section used to have, T75 0 

MEANS ^ 6^,7 7,2 
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or Strongly Disagreed was 7.2%. Clearly, the response to the system by those 
who used it was very positive. 

Effects On Productivity 

Once the system was developed and a baseline was established, the system 
was used to generate feedback. Next, goal setting was added to feedback, and 
finally incentives were added. 

The overall effects across the five units are shown in Figure 7, which 
plots the mean of the overall effectiveness of the five sections over time. As 
the figure indicates, overall effectiveness increased substantially over the base- 
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In order to calculate a single index of change in productivity, we calculated 
the percent gain from the baseline compared to maximum possible gain. That is, 
mean overall effectiveness was calculated from the baseline. The difference 
between this value and the maximum possible overall effectiveness the section 
could obtain was calculated. This maximum possible effectiveness was deter- 
mined by calculating the overall effectiveness score if the unit produced at the 
maximum possible value for each of the indicators. The gain in overall effec- 
tiveness from baseline to each treatment was then calculated, and expressed as 
the percentage of maximum possible gain. For example, assume that the baseline 
mean was 400 and the maximum possible overall effectiveness was 800. If the 
mean overall effectiveness during feedback was 600, this would be a gain of 
200. This is 50% of the maximum possible gain, and would be the value re- 
ported. This approach to calculating change is in some ways conservative, in the 
sense that the maximum possible increase is limited to 100%. 

Using this approach, the average increase over baseline was 50% for feed- 
back, 75% for goal setting, and 76% for incentives. Figure 8 presents similar 
data for Comm/Nav alone. The data show average increases of 30% for feed- 
back, 65% for goal setting, and 68% for incentives. Figure 9 presents the data 
for MS&D alone. Average percent increases were 54%, 77%, and 79%. Finally, 
Figures 10, 11, 12, and 13 show the productivity effects for each of the four 
sections of MS&D. 

These results indicate a major increase in productivity. The effects were 
extremely large. In addition, the MS&D effects were consistent across sections. 
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FIG. 7 PRODUCTIVITY 
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FIG. 12 PICKUP & DELI VERY 
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Potential Problems of Interoretattng 

Before we can confidently attribute these - effects to the experimental in= 
terventions several issues must be considered. -The first is the possible pres- 
ence of a Hawthorne effect, where productivity teould have increased simply be- 
cause the um^ts were singled out f the special tc^eatment of being in a research 
project. Although such an effect is indeed possib^le. the project was designed to 
a^i possib^ contamination from an effect. Specifically, the initial contact 
with the units during system development was quit= intensive, and if a Hawthorne 
effect was gomg to occur, it should have occufr-^d then. Since this was well 
before the start of the baseline, any productivity increase would have already oc= 
curred and would not contaminate the results. Thus, the presence of a 
nawmorne effect cannot explain the productivity increases. 

Another possibility was that the increases productivity which occurred 

during Uie treatments might have been due to increases in the number of person- 
nel m the units. For Comm/Nav, we looked at th-^ total number of persormel in 
the shop during each month of the effort. The mean number of persomiel by 

T^l ^'^^'^t ^ ^^'^ number of personnel during baseline 

was dU.y. This figure increased slightly during feedback to 33.0. was 32 8 dur- 
ing goal setting, and dropped to 31 J during incennrtives. Since per sonner levels 
aunng the period of highest gain in productivity were essentially equal to the 
level during baseline, we concluded that the increases in productivity were not 
caused by an increase in number of personnel, ^or MS&D. the data reviewed 
were^ total number of personnel, and the number hours of overtime logged per 
month. Unlike Comm/Nav, MS&D routinely had considerable overtime The 
number of personnel for MS&D by month is also presented in Figure 14 The 
mean was 51.8 for baseline, 53.7 for feedback, 4«.4 for goal setting, and 49 2 
for incentives. Thus, the overall number of personnel decreased over the period 
of ae treatments. The overtime data presented ir^ Figure 15 show that number 
of hours of overtime decreased during the tiois that productivity increased 
Overtime went from a mean of 1,348 hours per month during baseline to 892 
during^ feedback, 404 during goal setting, and 416 da^jing incentives. Thus by the 
end of the treatments, overtime was less than one— third of what it was during 

^fT: ^'""^^ '""^ °f P^roject. the productivity gains 

Uiat had occurred were made with no increase in number of persomiel in 
t^mm/INav. and a decrease in manpoyvsr in MS&D, 

Finally, it was possible that thcri could have teen changes occurring in the 
larger organizations, of which the five experimental units were part, which 
caused general increases in productivity for all unita.. To explore this possibility 
productivity data were collected on several sections that were similar to the 
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FIG. 14 PERSONNEL FOR 
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BY MONTH 
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experimental groups in the type of work they did. The idea was to compare 
changes m the productivity of these sections with changes in the five experimen- 
tal units. If the productivity of these control imits increased as much as that of 
the experimental units, then the experimental units' increased productivity was 
most likely not attributable to the interventions. 

Control group data for Comm/Nav consisted of ten measures of productiv= 
ity from eight maintenance units in the Component Repair Squadron, of which 
Comm/Nav was one section. The data were collected from baseline and put into 
a composite measure to express overall productivity of the control groups This 
composite measure was the sum of the ten measures. A plot of this composite 
IS presented in Figure 16. In Figure 16, the letter B. F. G. or I is used to indi= 
cate which months of control group data correspond to the treatments. The con- 
trol groups did not receive these interventions. 

The mean value across the ten Comm/Nav control group measures during 
baseline was 317. It dropped to m during feedback, and rose to 377 and 365 
during goal setting and incentives, respectively. These results show that the 
control groups decreased somewhat in productivity during the Comm/Nav feedback 
period, and increased thereafter. We believe, therefore, that the productivity in- 
crease during feedback in Comm/Nav was not due to wider organizational 
changes, smce the productivity of other squadron units actually decreased during 
this period. Furthermore, the increases during the Comm/Nav goal setting and 
incentives periods were not present across all units. These increases were 
brought aboiu primarily by large increases in productivity on two of the ten con- 
trol measures. 

Productivity data on four control measures were examined for MS&D 
"^^t^L^f reflected overall functioning of the Supply Squadron (exclusive 

of MS&D) and one other unit outside of Supply. None of these units was part of 
the interventions. The composite index is also presented in Figure 16. The 
mean of these four measures during baseline was 516,- it was 512 for feedback 
511 for goal setting, and 518 for incentives. Thus, there were essentially no 
changes m productivity for the MS&D control groups. 

In summary, the control group data indicate that the effects on productivity 
that occurred in the experimental units cannot be explained due to wider organi- 
zational changes in productivity. 

Having examined the possibility of Hawthorne effects, effects due to in- 
creases in the number of personnel, and effects caused by wider organizational 
changes, we believe it is safe to conclude that the changes in productivity that 
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occurred were attributable to the axperimental interventions. In addition, our 
subjective impressions and the reactions of the personnel in the units completely 
convinced us that this was the case. 

Indicator Data 

We have shown the jwsitive effects on overall affactiveness. Let us now 
examine the effects on the indicators themselves. Tables 2 and 3 present the 
mean values for each indicator, by treatment, for Comm/Nav and MS&D, respec- 
tively. For example, Table 2 shows that, on the average, there were 5,9% 
bounces (i.e., 5.9% of the items repaired were returned for reaccomplishment of 
repair) during the baseline period. During the incentives peri^, the average de- 
creased to 2*5%, 

In addition, the data for the most im^rtant indicators for each of the five 
units are plotted by month in Figures 17 through 22, These were the indicators 
judged most important by unit personnel, supervisors, and managers. For 
Comm/Nav the two most important indicators were percent of repair demand met 
and percent of bounces. For MS&D, the most Important indicators for the Re- 
ceiving, Storage and Issue, and Pickup and Delivery sections ware the speed with 
which they handled "priority 2" material. Priority 2 items were the most im- 
portant items they dealt with, usually a part for an aircraft that could not per- 
form its mission without that part. The most important indicator for Inspection 
was the number of aircraft parts that had not been inspected by the end of the 
day. The fewer the number of parts, the more effective the unit. 

These results indicate that the effects of the system were very powerfuL 
Inspection of the table indicates that major productivity increases occurred* The 
graphs show how dramatic some of the changes were. 

Feedback, Goal Setting, and Incentive Results 

We have presented results on the effects of feedback, goal setting, and 
incentives; however, it is instructive to focus on them more directly. Direct 
comparisons of the three treatments are presented in Figures 23 and 24. Figure 
23 is a plot of mean overall effectiveness by treatment. This figure presents 
two sets of results: overall effectiveness summed across all five sections 
(Comm/Nav and MS&D), and overall effectiveness across the four MS&D sec- 
tions alone. Fipjre 24 presents similar results separately for each of the five 
sections (Comm/Nav and MS&D). 
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Table 2. Comm/Nav Indicator Means by 

Treatment 



INDICATOR Baseline Feedback Goals Incentives 



% Bounces - * 


5.9 


4.5 


2.7 


2.5 


% QA Inspections Passed + 


91.9 


88.7 


90.0 


92.6 


Units Awaiting Maintenance - 


23.3 


18.8 


11.8 


10.4 


# Units Awaiting Parts - 


44.6 


39.0 


32.8 


36.9 


% Demand Met + 


88.5 


90.5 


93.9 


92.6 


# STS Tasks Completed + 


5.1 


6.4 


6.8 


8.6 


% Qual. Tasks, Comm + 


57.9 


^.9 


68.1 


69.4 


% Qual. Tasks, Nav + 


40.0 


42.5 


55.0 


60.1 


# Scheduled Trn. Overdue - 


1.3 


0.0 


0.0 


0.0 


# Mobility Eq. Overdue - 


.4 


0.0 


0.0 


0.0 


ff PimL Overdue - 


.9 


0.0 


0.0 


0.0 


ff 349 Errors - 


8.8 


1.2 


1.3 


1.7 


# Missed Appointments - 


.2 


1.0 


.2 


0.0 



1* A - after an indicator signifies that a smaller maan is higher productivity, 
a + after an indicator siEnif^es that a larger mean is higher productivity. 
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Table 3, MS&D Indicator Means by Treatment 



INDICATOR Baseline Feedback Goals Incentives 



Receiving 



In-chacking Errors/ ICW - * 




13 


0 


0 


Priority 2 Rec (Min) - 


118.6 


20.5 


16.2 


16.3 


r^riority 4 Kec (tiTS) ^ 


23 A 


5.3 


6.5 


4.3 


ff Whse Refusals in Receiving - 


2.8 


1.8 


0.0 


0.0 


ff Delinquent Rejects - 


137.6 


27.2 


1.9 


1.2 


Storage and Issue 










ff Inspection Findings - 


15.3 


19.0 


12.2 


9.4 


ff Whse Refusals Wrong Location 


- 2.5 


.8 


0.0 


.2 


% Cleared Off R36 + 


89.1 


96.5 


^.9 


98.3 


Priority 2 Iss (Min) - 


60.8 


13.4 


9.5 


8.9 


Priori^ 3 Iss (Min) - 


117.7 


22.3 


15.8 


20.9 


Priority 4 Iss (Hrs) - 


4.8 


10.5 


2.8 


2.7 


ff Repeat Findings - 


0.0 


0.2 


0.2 


0.0 


Pickup and Delivery 










# Delinquent Turn-ins - 


44.0 


23.8 


0.0 


0.1 


Priority 2 Iss (Min) - 


64.0 


35.3 


27.4 


27.1 


Priority 2 Rec (Min) - 


^.3 


42.0 


33.1 


34.2 


Priority 3 (Min) - 


76.5 


52.0 


35.9 


35.9 


Priority 4 (Hrs) - 


32.5 


20.3 


19.7 


16.6 


Vehicle Inspection Score + 


93.6 


94.6 


^.4 


95.5 


# Reportable Accidents - 


0.0 


0.0 


0.0 


0.0 


# Non- reportable Accidents - 


.3 


.4 


.2 


.0 



1. A ^ after an indicator signifias that a smaller mean is higher productivity, 
a -H after an indieator si^ifies that a larger mean is higher productivity » 



72 

62 



Table 3, (Concluded) 
MS&D Indicator Means by Treatment 



INDICATOR 
Inspection 

# Incoming Items Left - 
Dated Items Left - 

# Unidentified Items Left - 

# Aircraft Parts Left - 

# Functional Checks Left - 

# Suspect Items Left - 

# Late Monthly Inspections ■ 

# Returns From DP - 
% TCTOs Checked + 

# RODS From Other Bases 

# Off Base Shipments Left 

Branch Level Indicator 

# Delinquent Documents ~ 



Baseline Feedb nrJ: Gouh Incentives 



12.9 


5.4 


4.5 


6.0 


.3 


.4 


.1 


0.0 


17.6 


9.5 


2.2 


5.0 


2.2 


.7 


.3 


0.0 


.6 


.1 


.1 


.4 


.5 


0.0 


.1 


0.0 


4.3 


.4 


0.0 


.2 


1.0 


0.0 


0.0 


0.0 


DO.O 


99.8 


100.0 


100.0 


6.5 


6.8 


5.6 


7.4 


1.8 


.2 


.3 


.1 



306.6 139.2 60.0 48.2 
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FIG. 18 COMM/NAV 
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FIG. 24 AVERAGE EFFECTIVENESS 
BY TREATMENT-FIVE UNITS 
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It is clear from Figures 23 and 24 that a very consistent pattern resulted. 
There was a major effect due to feedback, a smaller but still large further in- 
crease during goal setting, and either no increase or a very small additional in^ 
crease during incentives. It is tempting to conclude that feedback increased pro- 
ductivity, goal setting increased it considerably more, and incentives havr little 
additional effect* Hov^evefi we do not believe this is the correct interprataiiun. 
The continued effects of feedback and ceiling effects must be considerec i i una 
interpratation of the results* These issues will be addressed in the Dibcus-lon 
section of this paper. 

It is also instructive to examine what the units did in terms of goal setting 
and incentives. The goal setting data over the ten months of goal setting are 
presented in Table 4. Difficulty of the self=set goals varied considerably across 
the five imits. This can be seen by the first column of the table, which indi- 
cates the number of goals (out of the ten possible) where the section's monthly 
goal was set lower than pr^uctivity had been for the previous month* The val- 
ues range from zero for the Receiving and Inspection sections to eight for the 
Storage and Issue section* Goal difficulty can also be seen from the second col- 
umn of the table, which shows how each section's monthly goals compared with 
their previous month's goals generally. That is, if the unit set monthly goals 
that werei on the average, 10% higher than their last month's pr^uctivity, the 
tabled value would be 110, As can be seen, most of the units set goals higher 
than their previous months' productivity. The one exception was the Storage and 
Issue section, which generally set goals about 5% below their productivi^ for the 
previous month. 

This difference in goal difficulty is also reflected in the third column of 
the table, which presents mean percentage of goal attained. If, on the average, 
the unit's productivity equaled its goal, the value would be 100; if productivity ex- 
ceeded the goal, the value would be above 100, The fourth column shows the 
number of months in which each unit achieved its goal. Both of these columns 
indicate that the Storage and Issue section fairly consistently exceeded its goals; 
Comm/Nav did so almost as well, and the rest of the units exceeded their goals 
less frequently* 

The data Indicate that all the units except Storage and Issue were gener- 
ally setting goals that actually represented increases in productivity. Even though 
there were several months when units set goals below their previous month's 
productivity, it was expected tiiat for some months this would be necessary* 
This usually occurred when a unit expect^ some change in workload or re- 
sources for the upcoming month. For example, if Pickup and Delivery had a 
driver on leave for the month, they would lower the goal for that month. 
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Table 4. Goal Data 



Number of 
Goals Set 
Below Last 
Month's 
Productivity 



UNIT 

Comm/Nav 2 

Receiving 0 

Storage & Issue 8 

Pickup & Dalivery 4 

Inspection 0 



Mean 

Percent Goal 
Was of Last 
Month's 
Productivity 

100.9 

103,2 

94J 

lOLl 

102.9 



Mean 

Percent of 

Goal 

Attained 

102.5 

97.2 

106,8 

99J 

99,4 



Number 
of Times 
Goal Was 
Attained 



2 
8 
3 
3 
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The results also suggest that the goals that were set were fairly difficult 
ones in each unit except Storage and Issue. Although the percent increase over 
the past month's productivity may seem low, this number includes those goals that 
were set below the past month's productivity due to workload or resource 
changes. In addition, all the units were producing at nearly their maximums, es- 
pecially during the last five to eight months of goal setting. Thus, even a small 
increase in productivity was difficult to achieve. 

Although Storage and Issue set substantially lower goals than did the other 
sections, this did not seem to have a negative effect on their productivity. Their 
percentage productivity increase over baseline during goal setting was 80%, com- 
pared to an increase of 77% for all four sections of MS&D combined. When in- 
centives were added to goal setting, their increase was 81% over baseline, while 
all of MS&D showed an increase of 79%. 

The incentive treatment was instituted for five months. During that time 
the frequency of earning incentives varied considerably among the units 
Comm/Nav received time-off least frequently. They received one half day and 
one full day over the five opportunities. The Receiving section and the Storage 
and Issue section each earned two half days and three full days off. Pickup and 
Delivery earned three half days and two full days. Inspection earned four full 
days and no half days. When time off was earned during a given month the 
units took that time off during the following month. If Comm/Nav. for example, 
earned time off for their February performance, they would take it sometime iii 
March. This applied to all units except Pickup and Delivery. This unit was un- 
dermanned and could not take their time off in the typical manner. Instead they 
took the time at a later date when the work load was less. 

Another point of interest relates to the way goal setting was conducted un- 
der the incentive system. Recall that goal setting was continued independent of 
incentives. Thus, as part of the incentive treatment, there was a level of over- 
all effectiveness established for the full and half days off; at the same time, 
there was a level of overall effectiveness set by each unit as a goal. We won- 
dered whether the goal setting would be superseded by tlie incentive system; that 
IS. whether the units would merely set goals at a level that would enable them to 
obtain one of the two incentives. Comparison of the goal data with the incentive 
levels indicates that there was no tendency for the two interventions to converge. 
The goals that were set were independent of the incentive levels. Out of the 25 
opportunities (5 units x 5 months) only 3 goals set by the units matched either of 
the incentive levels. In addition, observation of the goal setting sessions 
suggested that the goals were being set independently of those levels required for 
the incentives. 
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Also, there was no identifiable pattern to the goals in relation to the in- 
centive levels. In 8 of the 25 possible occasions, the goals were set above the 
highest incentive level (that needed to earn a full day off)* In 10 of the 25 oc- 
casions, the goals were set below the lowest incentive level (that needed for a 
half day off). For the remaining 7 occasions, the goals were set at or some- 
where between the level needed for the half day and full day off. 

Attitude Data 

Data were also collected on work attitudes. A questionnaire was admin- 
istered to incumbents and first line supervisors in each of the units. It mea- 
sured job satisfaction, turnover intentions, morale, individual role clarity,, clarity 
of objectives, and evaluation clarity. Job satisfaction was measured by seven 
items adapted from the Minnesota Satisfaction Questionnaire (Weiss, Dawis, 
England, & Loftquist, 1967), The actual items used for this and the remaining 
scales are listed in the appendix. The items for the morale scale were adapted 
from Institute for Survey Research (ISR) instruments (Seashore, Lawler, 
Mirvis, & Cammann, 1983). Items for individual role clarity were adapted from 
a Rizzo, House, and Lirtzman (1970) instrument to assess how well individuals 
believed they understood their roles witiiln the organization. Items for turnover 
intentions, clarity of objectives, and evaluation clarity were developed for this 
project. The clarity of objectives portion was designed to determine how well 
individuals believed they understood individual and organizational priorities. Fi- 
nally, the evaluation clarity portion was designed to assess how well they under- 
stood how a particular degree of pr^uctivity might be evaluated by a supervisor. 

The questionnaire was administered four times: during baseline, and again 
at the end of each treatment. Internal consistency reliability was assessed using 
Cronbach's alpha (Nunnally, 1978) calculated for each scale based on the data 
from the first administration. Internal consistencies were good for job satisfac- 
tion (.82), morale (.86), and for role clarity (.87). They were somewhat lower 
for evaluation clarity (,76) and for clarity of objectives (,64), and quite low for 
the two-item turnover intention scale (.39)* 

The treatment results are presented in Table 5. The means for the four 
conditions were analyzed with a one-way ANOVA, with the four administrations 
serving as a between-groups factor. A repeated measures analysis was not used 
since the number of individuals for all four administrations was very mall. 
The means for each treatment are indicated, along with the Ns, the p-value for 
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Table 5. Attitude Data by Treatment 



SCALE Baseline Feedback Goals Incentives £ MSW 
Job Satisfaction 

Means 23.48 25.33 24.23 22.82 .05 25.86 

Ns 60 70 56 45 

Turnover Intentions 

Means 6.9 6.7 6.7 7.0 .92 6.02 

Ns 70 71 54 47 

Morale 

Means 5.42 6.90 6.55 6.47 .01 7.67 

Ns 74 72 55 47 

Role niaritv 

Means 9.70 9.76 9.58 9.33 .73 4.53 

Ns 73 72 55 48 

Clarity of Obiectiyes ; 

Means 8.32 8.68 9.25 8.54 .08 4.25 

Ns 73 72 K 48 

Evaluation Claritv ' 

Means 7.13 6.20 4.79 5.11 .005 14.05 

Ns 67 55 48 36 



1. The lower the moan, the greater the evaluation clarity. For all other 
scales, the higher the mean, the more positive the attitude. 
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the between-administrations factor, and the mean square- within (MSW) error 
term for the between-administrations significance test. 

The results indicate that the measures of turnover intentions, individual 
role clarity, and clarity of objectives showed no significant changes, although the 
means for clarity of objectives became more positive, and the change was close 
to significant (p = .08). Job satisfaction, morale, and evaluation clarity became 
significantly more positive. In general, the data indicated that job attitudes under 
the treatments were as favorable or more favorable than before. 



Effects After The Departure Of The Research T 



eam 



Once the five-month incentive treatment was over, our on-base responsi- 
bilities officially ended. Although we were on base for a variety of purposes 
after this time, the units had no commitments to continue the system. To 
explore effects after our departure, we shall address the units' attempts to 
change the system, our training of unit personnel to take over the system, the 
tJnits' ability to run the system after we left and, fmally, the productivity scores 
attained since we left. 

A significant indication of the value of the system is that each of the five 
units elscted to continue the system after the researchers left. This meant that 
they would have to commit their own resources to put together the indicator data 
and run the programs producing the feedback reports. In addition, we were 
asked by both Comm/Nav and MS&D managers to develop the system in other 
units in their respective squadrons. Although we did not have the resources to do 
so, it did indicate the value that the units placed on the system. 

At the end of the formal incentives treatment, units were asked if they 
wished to modify the system. If they did want to, we would be there to assist 
if it became necessary. Comm/Nav and two sections of MS&D elected to make 
changes. In all three cases, the changes were made to eliminate indicators from 
the system. The indicators that were removed were either: ones with very flat 
contingencies, indicating they were not very important,- activities that the units 
were no longer going to perform; or indicators that were under such good control 
that they no longer needed to be measured. 

A major strength of the system is that it can accommodate changes read- 
ily. As changes occur in policies, procedures, or resources, changes will need to 
be made in the system. This can be done by eliminating indicators, redefining 
them, or altering the scaling of contingencies. Thus, the system can easily be 
altered to changing conditions. However, It must be understood that after such 
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changes are made, the new effectiveness scores are no longer comparable to the 
old scores. For example, if indicators are dropped, the same actual prc^uctivity 
will show up as lower overall effectiveness since some effectiveness points are 
lost due to the deleted indicators. This makes the inte^retation of effects over 
time difficult until a new "baseline" is established, A 'new baseline is estab= 
lished by taking the newly revised system of indicators and contingencies, and 
calculating its overall effectiveness for several months prior to the revision. 
For example, if some indicators are dropped, it is a straightforward matter to 
go back to the indicator data from prior months and calculate what overall ef- 
fectiveness would have been if those indicators had not been included* This then 
becomes the new baseline, and the effectiveness scores after ih& change can be 
compared directly to tiiis new baseline* If indicators are added, it is a matter 
of recalculating what the overall effectiveness for prior months would have been 
had these indicators been included. This is a simple matter, provided historical 
data are available. 

One feature of the system tfiat needed to be explored was whether the 
units would be able to use toe system after the researchers left. As part of 
that process, the units needed to be able to make changes in the system, since 
the need for changes would undoubtedly occur in the future. Thus, we were 
particularly interested in their ability to do this. In making these changes, it be- 
came clear that the management of the units understo^ the details of the sys- 
tem fairly well, and with help from the research team were able to make the 
changes. With this help, they were able to eliminate the indicators they wished, 
adjust the calculations of effectiveness to take the removal of the indicators into 
account, and recalculate past effectiveness to establish a new baseline. Our as- 
sessment was that Comm/Nav could make such changes completely on their own, 
and MS&D could do so with some assistance. 

Before we left the base, the units wanted us to train them to use the 
system, so that they could continue using it after we left. This proved to be a 
fairly simple task in 03mm/Nav, By the end of the incentives treatment, they 
had already taken over tiie collection of all the data that went into the system. 
They had only to be trained to use the microcomputer programs designed to cal- 
culate effectiveness scores and generate the feedback reports. This was done 
readily, and other than an occasional question, they operated the system com- 
pletely on their own. 

The training in MS&D was more involved. Although their intent to take 
over the system had been frequently expressed for some time, the actual com- 
mitment of personnel was not made until the end of the incentive treatment. 
Thus, tiie training could not be done gradually over many months, as would have 
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been optimal. In addition, the task of preparing the feedback reports in MS&D 
vvas more difficult than in Comm/Nav. The MS&D reports required the entering 
of data showing the amount of time it has taken to move property in the ware- 
house. Someone must take several hours each month to enter these data and run 
the program that calculates the mean times for the indicator data. During the 
mterventions, this was done by the research team. After our departure it had to 
be done by MS&D personnel. Thus, they had to learn how to enter these data 
and run the program, and also learn how to generate the feedback reports Be- 
cause of these factors, it took longer to train MS&D to take over the system, 
as of this writing they had taken over the system completely. 

A final consideration here is what happened to productivity after the units 
took over the system themselves. The results are shown in Figure 25 This 
figure not only indicates the effects after the units took over system operation 
but also demonstrates the capability of the system to generate a new baseline 

'^i'T^c?"**^ ^'"^^^ Comm/Nav and some sections 

of MS&D had deleted indicators from the system, we had only to recalculate the 
overall effectiveness scores back in time in order to develop a baseline for 
comparison. In this case, all changes in the system were made the month fol- 
lowing the incentive treatments, when our involvement in the interventions ended. 
To develop a baseline, we recalculated the overall effectiveness data for the five 
months of incentive treatment. This adjmtsd score is the overall effectiveness 
score that the units would have had during the incentive treatment if they had 
been under the revised system. 

Based on the revised system, the mean overall effectiveness score under 
incentives for Comm/Nav was 519, and the mean after they took over the system 
was 556. For MS&D, the mean under incentives was 1857, and the mean after 
ftey took over was 1792. For both units combined, the respective means were 
2376 and 2348 during and after incentives, respectively. Thus, the data indicate 
that productivity remained approximately as high after base personnel had respon- 
sibility for the system as it had been when the system was operated by the re- 
search team. 

IV. DISCUSSION 

The objectives of the present research were presented earlier in this pa- 
per as a series of research questions. We will repeat these questions and pre- 
sent the results pertinent to each. 

The first research question was whether the productivity measurement 
system could be effectively developed in an Air Force organization. The system 
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appears to be a very effective way of measuring productivity in the Air Force 
ts implementation is quite feasible, unit personnel were cooperative" in developing 
toe system and it showed good psychometric properties. In addition, system def 
vel^ment itself seemed to have a positive effect on unit functioning even before 
feedback was instituted. - oeiore 

Finally, the system appeared successful in aggregating productivity mea- 
sures across umts so that an integrated system could be developed across the 
four sections making up a branch. This process is actually quite simple once 
he b^sic sys em is developed in each section. The application of this aggrega^ 
tion to much larger and more complex organizational units seems quite feasible 

The development of the system and its positive effects worked well on 
imits to were quite differem from one another. The units differed greatly in 
^d MS&D f"°'''i ^^^^^^ doing repair of sophisticated equipment, 
and MS&D operating a large warehouse. The technologies were quite differen 
between the two units, as well as between the four sections of MS&D There 
were also great differences in the type of organizational structure and tiie work 
flow. The umts varied considerably in size and the personnel varied consider- 
ab^ m academic as well as in technical education. They also differed in their 
imtial levels of performance. Yet with all these differences, the system was 
developed and worked extremely well in each unit. This adds considerable sup- 
port to the generahzability of the approach. 

The second research question was whether using the feedback resulting 
from the productivity measurement system would increase productivity Feedback 
had a very strong effect on productivity. An average gain in productivity of 50% 
occurred across the five units during feedback. This increase occurred with 
either no change or a decrease in manpower. Competing explanations for the 
positive results, such as Hawthorne effects, increases in manpower, and in= 
creases brought about by other organizational changes, can be ruled out. 

The third research question was whether goal setting and incentives would 
increase productivity over feedback. At first inspection, the answer seems to be 
that goal setting adds to feedback, but incentives add little beyond feedback plus 
goal setting This io.pression is based on the overall changes in productivity, 
where feedback resulted in a gain in productivity of 50% over baseline- feedback 
plus goal setting, 75%; and feedback plus goal setting and incentives, 76%. 

We do not, however, believe that this is the correct interpretation In- 
spection of Figure 7 shows that across the five units, there was a large change 
when feedback was instituted. The early months of feedback showed a very 
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strong improvemant in productivity. This improvement ttien began to slow down, 
and finally, the curve flattened out during the last months. The negatively accel^ 
erated ourve from the start of feedback to tiie end of incentives looks very 
much like a learning curve. This suggests that what may have been happening 
was that the imits were continuing to learn how to improve productivity solely on 
the basis of the feedback. The possibility exists tiiat the same amount of in- 
crease over time would have occurred if only feedback had been used. One bit 
of supporting evidence for tiUs interpretation is that there was no ^mp in pro- 
ductivity at the institution of either goal setting or incentives. One would have 
expected a noticeable increase in the month these systems were started if they 
were indeed adding much to productivity, 

A second and very different interpretation has to do with a ceiling effect. 
One could argue that since the imits were near their maximum possible levels of 
overall effectiveness by the second month of goal jetting, further increases were 
not really possible. Thus, both goal setting effects and incentive effects would 
have been stronger if there had been more rw>m for improvement. Although this 
interpretation is possible, a counter arpiment would be that if productivity cm be 
increased to its maximum with only feedback or feedback plus goal setting, in- 
centives are simply not necessary. It is also possible, however, that goal setting 
and/or incentives would be necessary to sustain the high levels of productivity 
over time. 

There is no conclusive way of deciding among these different interpreta- 
tions of the data. The only way to make such conclusions would be to replicate 
the study using a much longer period for each treatment. Our own best estimate 
is that goal setting added somewhat to feedback^ but not a great deal, and that 
incentives did not add furtiier. This conclusion is based on the learning curve 
shape of the productivity data, the lack of a jump in prDuuctivity at the institution 
of goal setting and incentives, and the subjective impressions of the unit mem- 
bers. 

The fourth research question was whether the system would change work 
attitudes. The results indicate that work attitudes either improved or stayed the 
same across a series of attitude dimensions. 

A fifth research question was whether the system would be seen as valu- 
able to the users. Clearly, it was. Attitudes toward the system ranged from 
positive to very positive. The members of tfie units were quite proud of the 
system and tiieir Improvements. As one indication of this, unit members fre- 
quently showed the system to people visiting the unit. In addition, by the end of 
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ff ^' of "nits continued using the system on their own. 

and the managers of all the units wanted to expand it to other units. 

The final research question was whether the system would operate suc- 

^,1" f 11 'T'*"'' ''''^'"^ The system has operated 

successfully, at least for the five months for which we have data at this writ- 
ing. Umt personnel learned to operate the system successfully, they have contin- 
ued_ Its use and the units have continued to maintain their productivity at the 
highest level that it reached during the interventions. 

VI. CONCLUSIONS 

A number of conclusions can be drawn from this research. First we 
will discuss specific research conclusions concerning the capabilities of the 'sys- 
tem. We will then present our conclusions concerning the productivity measure- 
ment system development process, and conclusions drawn about implementing 
feedback, goal setting, and incentive interventions. 

Conclusions About Pr o ductivitv Measurement System Capabilities 

The productivity measurement system that was evaluated in the present ef- 
fort seems to be a very successful method of measuring productivity in Air 
Force organizations. Its use as the basis for a feedback system produced very 
large increases in productivity. Whether goal setting adds to productivity beyond 
feedback alone is not clear. Essentially, incentives did not increase productivity 
above feedback plus goal setting. The productivity interventions produced either 
an improvement or no change in various job attitudes. The units liked the sys- 
tem, and continued to use it with equal success after the departure of the re- 
search team; their high productivity has been maintained. Specifically the posi- 
tive productivity results continued for the 15 months that the research' team was 
present on base, and have continued for at least 5 months after that. 

The development and application of the system also has a number of added 
benefits. It allows for the direct comparison of the productivity of different 
umts to each other. It can be used with both effectiveness and efficiency ap- 
proaches to productivity. In addition, it can be applied to any level of the organi- 
zation, allows for competition among different units, helps identify the priorities 
for mcreasing productivity, and serves as an excellent basis for evaluating any 
changes made in the organization. 
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Conclusions Atout the Process of Developing Productivity Systems 



In doing this research, a number of conclusions were drawn concerning the 
process to be used for developing productivity measurement and enhancement pro- 
grams in Air Force environments. 

One is the importance of having the personnel w^ho are going to be using 
the system be heavily involved in its development. The perception of unit per- 
sonnel was that some previous programs imposed from, above had not been ef- 
fective because these programs were not designed with an appreciation for their 
unique needs and environmentp It is much more effective to have hea%y Involve- 
ment from the personnel that are going to use the system so that the final prod- 
uct will fit their needs and they do not feel that it is yet one more project im- 
posed from above. 

It also seems more effective to develop such programs from the bottom of 
the organization up. The lower levels of supervision know the most about tiie 
functioning of the imit^ the real critical issues* what are reasonable goals, md 
what are attractive incentives. In addition, these are the people that will make 
the system work. It is important to have their involvement and knowledge from 
the start. It is also important to have higher-level Involvement to approve the 
system* This should be done as the measurement system is being developed, not 
delayed until the system is complete. We believe tiie techni^e of getting ap- 
proval on products and indicators before starting contingency development proved 
very valuable. It not only helped clarify policy earlier in the process* but also 
helped prepare everyone for the eventual implementation. This approach gave all 
levels of the organization a chance to learn about the system as it was being de- 
veloped so that they would know how to use it when it was finished. It also 
served to generate considerable eagerness at all levels to receive the first feed- 
back from the system. 

We also learned the importance of using a multiple meeting stmcture that 
allows for an iterative approach to the development of toe system. Our strategy 
was to summarize the results of each meeting and present them at tiie next 
meeting to assure consensus was reachedp In tins way, imit persoiuiel had time 
to think about the issues, discuss them among themselves, wd be prepared to ap- 
proach the question with fresh perspectives at the next meeting. Personnel need 
time to adjust to the idea of completely capturing what they ^ in a productivity 
measurement system. They need to think about how to structure a feedback re- 
port, how many levels of incentives to use, etc. They need to consider what the 
implications are, and be able to discuss the issues among ttiemselves thoroughly. 
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n.ny, fteir sense of ownership of the system was increased by their^efense of 
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due, and so that the positive and negative information are |iv?-en appropriate em- 
phasis. 

It is extremely important that prc^rams such aft th^se used here have 
higher management support. This is very important for the contiiiued operation 
of the program. In this regard, it is very important that se^or management be 
kept informed about the program and fully understand it. 

We did not anticipate the difficulty of keeping higher ananagement support 
of the project when there was turnover in these manageiaea-t ^sitions. There 
was generally an initial resistance to the program by the nev^ manager. In ret- 
rospect, this is not surprising. The program was not develo>«d by the new man- 
ager, and he/she did not have a sense of ownership of it. Therefore, it is very 
important to take considerable time witii the new manager sm that he/she can be 
convinced that the program is indeed a sound one. 

There were also several issues associated with goals and incentives. We 
have already discussed the issue of "reportable" and "noii-reportable" goals, the 
importance of participation in the design of the systemi and the importance of 
joint supervision and incumbent participation in setting goal levels. For incen- 
tives, we have discussed the importance of the number of incentive levels to be 
used and of ensuring equity across units in what it takes to oibtain the incentives. 

Another issue regarding incentives is that one should expect resistance to 
the use of incentives, especially from supervisors. In the pr-esent effort, we en- 
countered considerable resistance from supervisors. The i»ajor point made by 
those who were opposed to incentives was that Air Force per-sonnel should not be 
rewarded for simply doing what they are already supposed to do. There was 
also some indirect evidence that some supervisors felt such an incentive system 
would undermine their power and their own prerogatives to reward individuals 
and units informally. In contrast, the incumbents were Biuch^ more positive about 
the incentive system than were supervisors. Incumbents wanted the time-off, but 
more importantly, they wanted some tangible form of recOEiiition for their high 
productivity. There were, of course, some supervisors that were in favor of 
incentives and some incumbents who were not. We wool« estimate that about 
30% of the supervisors and 85% of the incumbents were in f».vor of incentives. 

Finally, care should be taken with an incentive system to ensure that man- 
agement does not increase the level of performance needed to obtain an incentive 
once productivity has increased. This practice could easily lead to resentment 
and a sense of inequity among unit personnel. In one unit, there was an attempt 
to do tius and tiie research team managed to convince the manager that this 
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evILamy!'''* P"*"" applications should be prepared for this 

Summary 

In conclusion., tha present effort has shown that productivity in units with 
complex Air Force jobs can be measured with this approach. The approach 
proved very successful in this application and has many advantages that make it 
attractive for otaaer applications as well. The use of group^level feedback re= 
suiting from the productivity measurement system produced large increases in 
productivity, md may well have accounted for the majority of the overall produc- 
tivity gain. Goal setting seemed to add to productivity, but incentives added lit- 
tie beyond feedback and goal setting. The interventions of feedback and goal 
setting clearly have a place in the techniques of productivity enhancement in the 
Air rorce. 
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JOB SATISFACTION 



Response Format: Five-point Likert format with anchors of Very Dissatisfied 
Dissatisfied, Not Sure, Satisfied, and Very Satisfied. 

Items: 

1. The chance to do something that makes use of my abilities. 

2. The way Air Force policies are put into practice. 

3. The freedom to use my own judgment. 

4. The chance to try my own methods of doing the work. 

5. The working conditions, 

6. The praise I get for doing a good job. 

7. The feeling of accomplishment I get from the job. 



MORALE 

Response Format: Five-point Likert format with anchors of Strongly Disagree 
Disagree, Neutral, Agree, and Strongly Agree. 

Items: 

1. A spirit of teamwork exists between the people in my section. 

2. A spirit of teamwork exists between the people in my branch. 

3. The people in the section work together to accomplish the section's objectives. 

4. The people in the branch work together to accomplish the branch's objectives. 

5. I feel a sense of pride at being a member of this section. 

6. I feel a sense of pride at being a member of this branch. 



TURNOVER INTENTIONS 

Response Format: Five-point Likert format with anchors of Strongly Di 
Disagree, Neutral, Agree, and Strongly Agree. 

Items: 

1. I plan to reenlist or extend. 

2. I would like to leave the Air Force next year. 
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ROLE CLARITY 

Response Format: Five-point Likert format with anchors of Strongly Disagree 
Disagree., Neutral, Agree, and Strongly Agree. ' 

Items: 

1. I know what my responsibilities are. 

2. I know exactly what is expected of me. 

3. Explanation is clear of what has to be done. 



CLARITY OF OBTECTIVES 

Response Format: Five-point Likert format with anchors of Strongly Disagree 
Disagree, Neutral, Agree, and Strongly Agree. 

Items: 

1. I understand which of my work objectives are more important than others 
2,. 1 he work objectives of my section are clear and specific. 

3. The work objectives of the branch are clear and specific 

4. I understand which of my section's objectives are more important than others 

5. I understand which of the branch objectives are more important than others ' 
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EVALUATION CLARITY 



Instructions: For each situation below, please circle your estimate of the lowest 
and highest ratings you think your supervisor might give to you or your work 
group. For some of these duties you may be very sure about your supervisor s 
evaluations, and the "lowest" and the "highest" ratings would be very close. For 
other situations, you might be imsure what kind of evaluation your supervisor 
would give; therefore, there would be a big difference between the highest and 
lowest possible evaluations. Please rate what you believe your supervisor's eval- 
uation of you and your work group would be for the following monthly perfor- 
mance data. 

Please use the following in deciding which number to circle. (The specific items 
were tailored to be relevant to each section.) 

1 = Worst possible 2 * Fmr 3 = Below average 4 - Average 
5 = Above average 6 - Good 7 - Excellent 



1. On the average, 
75 percent of the 
unfrozen rejects 
were cleared in a day. 

2. MS&D took an average 
of 2 hours to get all 
priority 2 DOR items 
to the custoniers. 

3. MS&D took an average 
of 2 hours and 20 
minutes to get all 
priority 3 items to 
the customers. 

4. MS&D took an average 
of 48 hours to get 

all priority 4 items 
to the customers. 

5. There was a daily 
average of IM 
delinquent documents 
charged to MS&D per 
month. 
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